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@ Sound field and sound image control apparatus and method. 



@ The apparatus of the invention calculates filter 
coefficients for controlling sound field and sound 
image, based on a plurality of first impulse response 
signals and a pair of second impulse response sig- 
nals. The plurality of first impulse response signals 
indicate impulse responses from loudspeakers re- 
producing audio signals to both ears of a listener. 
^ The pair of second impulse response signals in- 
_ dicate impulse responses from a reference loud- 
speaker at a position at which a sound image is 
^ localized to both ears of the listener. The apparatus 
^ includes: a feature extracting section for receiving 
00 the pair of second impulse respons signals, for 
CD 



extracting parameters representing features of the 
pair of second impulse response signals, and for 
outputting parameter signals; a signal adjusting sec- 
tion for adjusting at least one of the plurality of first 
impulse response signals based on the parameter 
signals, and for outputting a pair of third impulse 
response signals having the same features as the 
extracted features; and a coefficient calculating sec- 
tion for calculating the filter coefficients for control- 
ling the sound field and sound image, based on the 
plurality of first impulse response signals and the 
pair of third impulse response signals applied from 
the signal adjusting m ans. 



UJ 



EP 0 684 751 A1 




P 9 q 

58-2 58-1 58-3 



1 



EP 0 684 751 A1 



2 



BACKGROUND OF THE INVENTI ON 

*. . . — . 

1 . Reld of the Invention: 

The present invention relates to a sound field 
and sound image control apparatus and a sound 
field and sound image control method for perform- 
ing audio reproduction with presence in audiovisual 
equipment. More particularly, the present invention 
relates to a filter coefficient calculating apparatus 
and a filter coefficient calculating method for per- 
forming the control sound field and sound image. 

2. Description of the Related Art: 

Recently, movies and the like are more fre- 
quently enjoyed at home because the use of video 
tape recorders (VTRs) and the like is wide spread, 
so that even a small-scale audiovisual (AV) system 
for home use is desired to perform audio reproduc- 
tion with presence. A private room in the house or 
the like generally Involves limitations such as room 
space and equipment. In many cases, additional 
loudspeakers for sound control or surround-sound 
reproduction cannot be located in the rear and the 
side of a viewer. For such cases, a technique has 
been developed for performing stereophonic sound 
image control and sound field reproduction with 
presence only by using general 2 channels (2-ch) 
loudspeakers, or 2-ch loudspeakers accommodated 
In a TV set (for example, see JAS journal, Septem- 
ber 1990). 

A conventional sound field and sound image 
control apparatus using 2-ch reproducing loud- 
speakers will be described below. 

Figure 14 schematically shows a conventional 
sound field and sound image control apparatus 800 
and a method for localizing the sound image in the 
left rear of a listener 86 by the conventional ap- 
paratus 800. 

In the apparatus 800. sound source signals S- 
(n) generated by a sound source 81 are processed 
by finite impulse response (FIR) filters 82-1 and 82- 
2, and then the processed signals are reproduced 
from a left-channel (L-ch) reproducing loudspeaker 
83 and a right-channel (R-ch) reproducing loud- 
speaker 84, respectively. For the FIR filter 82-1, 
filter coefficients (impulse responses) H1(n) are set. 
For the FIR filter 82-2. filter coefficients H2(n) are 
set. In cases where the apparatus 800 is used for 
digital processing, an A/D (analog-to-digital) con- 
verter and a D/A (digital-to-analog) converter are 
required. For simplicity, such converters are omit- 
ted in the figure. The listener 86 stays at a position 
distant from the two loudspeak rs 83 and 84 by 
equal distances (i.e., on the center line), and faces 
the front (i.e., faces toward the middle point be- 
tween two loudspeakers). 



In Figure 14, C1(n) indicates an impulse re- 
sponse from the L-ch loudspeaker 83 at the posi- 
tion of the left ear of the listener 86 (to be more 
accurate, the position of the eardrum; and in the 

5 actual measurement, it is measured at the entrance 
of the auditory canal when an impulse is input to 
the loudspeaker 83). Similarly, C2{n) indicates an 
impulse response from the L-ch loudspeaker 83 at 
the position of the right ear of the listener 86. C3(n) 

70 indicates an impulse response from the R-ch loud- 
speaker 84 at the position of the left ear of the 
listener 86. and C4(n) indicates an impulse re- 
sponse from the R-ch loudspeaker 84 at the posi-. 
tion of the right ear of the listener 86. In addition. 

75 T1(n) and T2(n) indicate impulse responses from a 
reference loudspeaker 85 to the left and right ears 
of the listener 86, respectively. The respective val- 
ues of C1(n) - C4(n). T1(n) and T2(n) can be 
obtained by actual measurements or simulation. 

20 These S(n), Ci(n) (i = 1 to 4), T1(n). and T2(n) 

are represented as discrete-time signals with a 
finite length. That is, n actually means nT in which 
a certain short time (sampling time) T is used as a 
unit. Herein, in order to provide the description in 

25 time domain, the impulse responses are used. For 
frequency domain, the same description as in the 
case of time domain can be expressed by using 
transfer functions obtained by Fourier-transforming 
the impulse responses. 

30 With the above construction, if the sound 

source signals S(n) which are impulse signals are 
input, and they are reproduced from the L-ch re- 
producing loudspeaker 83 and the R-ch reproduc- 
ing loudspeaker 84, the impulse response char- 

35 acteristic L(n) at the left-ear position of the listener 
86 and the impulse response characteristic R(n) at 
the right-ear position (i.e., the head-related transfer 
functions in time domain) are expressed as follows: 

40 L{n) = H1(n)-C1(n) + H2(n)'C3(n) (1) 

R(n) = H1(n)*C2(n) + H2(n)*C4(n) (2) 

where the symbol * indicates a convolution. 

In general, if two pairs of the head-related 

45 transfer functions are equal to each other, it may 
be assumed that each sound represented by the 
respective pair of transfer functions is perceived by 
the listener as coming from the same direction. 
Accordingly, if the filter coefficients H1(n) and H2- 

50 (n) are set so that L(n) and R(h) become equal to 
T1(n) and T2(n), respectively, the listener 86 can 
feel (perceive) that the sound image is localized at 
the position of the reference loudspeaker 85, by 
reproducing the sound source signals S(n) with 2- 

55 ch loudspeakers located in front of the listener 86. 

The above-mentioned convolution operation is 
performed by the FIR filters 82-1 and 82-2. Rgure 
15 shows the basic construction of each of the FIR 
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filters 82-1 and 82-2. As is shown in Figure 15. the 
FIR filter has an input terminal 91 for inputting a 
signal, and N delay elements 92 each for delaying 
a signal by a time t which are connected in series. 
On both ends of the series of d lay elements 92, 
and between respective two delay elements 92, 
multipliers 93 are connected, respectively. Each 
multiplier 93 multiplies an input signal by a filter 
coefficient, which is referred to as a tap coefficient, 
and outputs the resultant signal to an adder 94. 
The signal obtained by the addition in the adder 94 
is output from an output terminal 95. 

In general, for such an FIR filter, a dedicated 
LSI such as a digital signal processor (DSP), which 
performs multiplication and addition at a high 
speed, is used. In the multipliers 93. the impulse 

responses h(i) (i = 0 N) are set as the tap 

coefficients. A delay time t corresponding to a 
sampling frequency at the conversion of an analog 
signal into a digital signal is set in the delay ele- 
ment 92. The multiplication and delay are repeat- 
edly performed to input signals, and they are ad- 
ded to each other and then output. Thus the con- 
volution operation is performed. 

The above description is made for digital sig- 
nals, so that, in the actual implementation, an A/0 
converter is required to convert an analog signal 
into a digital signal before inputting the signal to 
the FIR filter, and a D/A converter is required to 
convert the output digital signal into an analog 
signal. However, the converters are not shown in 
Rgure 15, 

Figure 16 shows a conventional exemplary de- 
vice for calculating filter coefficients to localize a 
sound image. From the reproduction-system char- 
acteristics input terminals 901 - 904. signals cor- 
responding to the reproduction-system impulse re- 
sponses Cl(n)-C4(n). which represent the charac- 
teristics of the reproduction system, are input, re- 
spectively. From the reference characteristics input 
terminals 905 and 906, signals corresponding to 
the impulse responses T1(n) and T2(n), which re- 
present the reference characteristics, are input, re- 
spectively. These input impulse response signals 
are all input into a filter coefficient calculator 910. 

When the impulse response signals of the re- 
production-system (Cl(n) - C4(n)) are applied, the 
filter coefficient calculator 910 calculates filter co- 
efficients H1(n) and H2(n) for localizing a sound 
Imag (hereinafter referred to as sound image lo- 
calization coefficients) so that the reference char- 
acteristics become the impulse r sponses T1(n) 
and T2(n) (specifically, a matrix operation is per- 
formed in the filter coefficient calculator 910). The 
filter CO fficient calculator 910 calculates candi- 
dates H*1(n) and H'2(n) for H1(n) and H2(n) which 
satisfy the right sides of Equations (1) and (2) 
above. The calculated candidates H'1(n) and H*2(n) 



are output to a filter coefficient setting device 920 
together with the reproduction-system impulse re- 
sponse signals CI (n) - C4(n). 

The filter coefficient setting device 920 sets the 

5 impulse responses H*1(n) and H'2(n) for FIR filters 
941 and 942, respectively, and sets the impulse 
responses C1(n) - C4(n) for FIR filters 931 - 934. 
respectively, as tap coefficients. 

When the setting of tap coefficients is com- 

10 pleted, the impulse generator 950 generates an 
impulse signal. The impulse signal is processed by 
convolution in the FIR filters 941 and 942. and the 
FIR filters 931 - 934, added by adders 961 and 
962, and then output, as is shown in Rgure 16. 

75 These operations are equivalent to the operations 
indicated by the right sides of Equations (1) and (2) 
which are performed by using H*1(n) and H'2(n) 
instead of H1 (n) and H2(n). 

The output of the adder 961 is compared with 

20 the impulse response T1(n) of the reference char- 
acteristic by a subtracter 971. The output of the 
adder 962 is compared with the impulse response 
T2(n) of the reference characteristic by a subtracter 
972. 

25 The outputs of the subtracters 971 and 972 - 
(indicative of differences between the reproduction 
characteristics and the reference characteristics) 
are input into a feedback controller 980. The feed- 
back controller 980 instructs the filter coefficient 

30 calculator 910 to repeatedly perform the operation 
until the absolute values of the signals from the 
subtracters 971 and 972 become smaller than a 
predetermined positive value. The filter coefficient 
calculator 910 repeats the operation using T1(n) 

35 and T2(n) which are delayed by a predetermined 
time. 

When the absolute values of the output signals 
of the subtracters 971 and 972 become smaller 
than the predetermined positive value, the opera- 

40 tion of the filter coefficient calculator 910 is stop- 
ped. Then. HM(n) and H'2(n), which are obtained at 
that time, are output from output terminals 907 and 
908, as the valid H1(n) and H2(n). 

When the sound image localization coefficients 

45 H1(n) and H2(n) which are thus obtained are set in 
the sound image localization device and the re- 
production is performed, a sound image can be 
localized at a position where a loudspeaker does 
not actually exist. In addition, if a sound image is 

50 localized in an expanded region, as compared with 
the actual loudspeaker positions with respect to the 
listener, it is possible to perform audio reproduction 
with expansion and presence. 

However, in the prior art described above, the 

55 filter coefficients H1(n) and H2(n) are set for the 
listener 86 who stays on the center line. Accord- 
ingly, when the listener 86 moves away from the 
center line during the reproduction of the sound 
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source signals S(n), and when a plurality of listen- 
ers exist, the advantages of the sound innage con- 
trol are drastically deteriorated for the listeners who 
are located at positions away from the center line, 
for the following reasons. 

The impulse responses from the loudspeaker 
positioned in front of the listener 86 are usually 
largely different from the impulse responses from 
the loudspeaker positioned at the rear of the lis- 
tener 86, so that the filter coefficients H1{n) and 
H2(n) have frequency characteristics with large 
peaks and dips, In order to realize T1 (n) and T2(n) 
by using C1(n) - C4(n). Therefore, when the posi- 
tion of the listener 86 is changed slightly, the 
impulse responses from the reproducing loud- 
speakers 83 and 84 to the listener are significantly 
varied. Accordingly, a problem associated with 
such a conventional technique is that the service 
area (an area to which good sound image control 
can be performed) is limited and small. 

The method for calculating the filter coefficients 
in the above conventional technique has no prob- 
lem in theory. However, in practice, if the position 
of the listener 86 is slightly changed, the impulse 
r sponses are significantly varied and it is difficult 
to correct the deviations in higher frequency ranges 
in particular. Therefore, a problem exits in that the 
quality of the sound reproduced from loudspeakers 
83 and 84 is different from that of the sound 
actually reproduced by the reference speaker 85. 
This causes the deterioration of the sound quality 
of the sound image localized by the conventional 
device 800. 

SUMMARY OF THE INVENTION 

The apparatus of this invention calculates filter 
coefficients for controlling sound field and sound 
image, based on a plurality of first impulse re- 
sponse signals and a pair of second impulse re- 
sponse signals, the plurality of first impulse re- 
sponse signals indicating impulse responses from 
loudspeakers reproducing audio signals to both 
ears of a listener, the pair of second impulse re- 
sponse signals indicating impulse responses from 
a reference loudspeaker at a position at which a 
sound image is localized to both ears of the lis- 
tener. The apparatus Includes: a feature extracting 
section for receiving the pair of second impulse 
response signals, for extracting parameters repre- 
senting features of the pair of second impulse 
response signals, and for outputting parameter sig- 
nals; a signal adjusting section for adjusting at least 
one of the plurality of first impuls response sig- 
nals based on the parameter signals, and for out- 
putting a pair of third impulse response signals 
having the same features as the extracted features; 
and a coefficient calculation section for calculating 



the filter coefficients for controlling the sound field 
and sound image, based on the plurality of first 
impulse response signals and the pair of third 
impulse response signals applied from the signal 

5 adjusting section. 

In one embodiment of the invention, the coeffi- 
cient calculation section sets the filter coefficients 
so that the pair of third impulse response signals 
are substantially equal to a pair of fourth impulse 

10 response signals, the pair of fourth impulse re- 
sponse signals indicating a pair of impulse re- 
sponses at both ears of the listener when impulse 
signals are reproduced from the reproducing loud- 
speakers. 

75 In another embodiment of the invention,* the 
apparatus further includes: a response characteris- 
tic calculation section for calculating a pair of im- 
pulse responses at both ears of the listener when 
the impulse signals are reproduced from the re- 

20 producing loudspeakers, based on the first impulse 
response signals and the filter coefficients, and for 
outputting the pair of fourth impulse response sig- 
nals; a comparison section for comparing the pair 
of fourth impulse response signals with the pair of 

25 third impulse response signals, and for outputting a 
correlation signal; and a control section for output- 
ting a control signal which controls the coefficient 
calculation section, based on the correlation signal, 
wherein, in accordance with the control signal, the 

30 coefficient calculation section selectively performs 
one of two operations, in one operation signals 
indicative of the calculated filter coefficients are 
output, and in the other operation the filter coeffi- 
cients are again calculated using signals which are 

35 obtained by delaying the pair of third impulse re- 
sponse signals by a predetermined time. 

In another embodiment of the invention, the 
feature extracting section includes: a level ratio 
detection section for receiving the pair of second 

40 impulse response signals, for detecting a level ratio 
a of the pair of second impulse response signals, 
and for outputting a level ratio detection signal; and 
a time difference detection section for receiving the 
pair of second impulse response signals, for de- 

45 tecting a time difference dt of the pair of second 
impulse response signals, and for outputting a time 
difference detection signal. 

In another embodiment of the invention, the 
signal adjusting section includes: a selecting see- 
so tion for selecting a pair of first impulse response 
signals from among the plurality of first impulse 
response signals; a time difference adjusting sec- 
tion for receiving the selected pair of first impulse 
response signals and th time difference detection 

55 signal, for adjusting the selected pair of first im- 
pulse response signals so that a relative time dif- 
ference of the pair of first impulse response signals 
is equal to the fime difference dt based on the time 
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difference detection signal, and for outputting a pair 
of adjusted impulse response signals; and a level 
ratio adjusting section for receiving the pair of 
adjusted impulse response signals and the level 
ratio detection signal, for adjusting a gain of the 
pair of the adjusted impuls response signals so 
that the level ratio of the adjusted impulse re- 
sponse signals in the pair is equal to the level ratio 
a based on the level ratio detection signal, and for 
outputting the pair of gain-adjusted signals as the 
pair of third impulse response signals. 

In another embodiment of the invention, the 
signal adjusting section includes: a selecting sec- 
tion for selecting one first impulse response signal 
from among the plurality of first impulse response 
signals; a time difference adjusting section for re- 
ceiving the selected first impulse response signal 
and the time difference detection signal, for delay- 
ing the selected first impulse response signal by 
the time difference dt based on the time difference 
detection signal, and for outputting a delayed im- 
pulse response signal; and a level ratio adjusting 
section for receiving the delayed impulse response 
signal and the level ratio detection signal, for ad- 
justing a gain of the delayed impulse response 
signal by multiplication of the delayed impulse re- 
sponse signal by the level ratio a based on the 
level ratio detection signal, and for outputting an 
adjusted impulse response signal. Also, the pair of 
third impulse response signals are constituted of 
th selected first impulse response signal and the 
adjusted impulse response signal. 

In another embodiment of the invention, the 
feature extracting section is a transfer characteristic 
detection section for receiving the pair of second 
impulse response signals, for detecting transfer 
characteristics of the pair of second impulse re- 
sponse signals, for calculating a transfer character- 
istic ratio, and for outputting a characteristic ratio 
signal. 

In another embodiment of the invention, the 
signal adjusting section includes: a selecting sec- 
tion for selecting one first impulse response signal 
from among the plurality of first impulse response 
signals; and a transfer characteristic adjusting sec- 
tion for receiving the selected first impulse re- 
sponse signal and the characteristic ratio signal, for 
adjusting a transfer characteristic of the selected 
first impulse response signal based on the char- 
acteristic ratio, and for outputting an adjusted im- 
pulse response signal. Also, the pair of third im- 
pulse response signals are constituted of the se- 
lected first impulse response signal and the ad- 
just d impulse r sponse signal. 

In another embodiment of the invention, th 
transfer characteristic detection section includes: a 
first transform section for transforming the received 
pair of second impulse response signals into a pair 



of first characteristic signals represented in fre- 
quency domain; and a first calculation section for 
calculating a transfer characteristic ratio of the pair 
of second impulse response signals based on the 

5 first characteristic signals, and the transfer char- 
acteristic adjusting section includes: a second 
transform section for transforming the selected first 
impulse response signal into a second characteris- 
tic signal represented in frequency domain; a sec- 

70 ond calculation section for multiplying the second 
characteristic signal by the transfer characteristic 
ratio indicated by the characteristic ratio signal; and 
an inverse transform section for transforming the 
multiplied signal into a signal represented in time 

75 domain. 

In another embodiment of the invention, the 
first and second transform sections are Fourier 
transform sections, and the inverse transform sec- 
tion is an inverse Fourier transform section. 

20 According to another aspect of the invention, 
the sound field/sound image control apparatus per- 
forms a sound field control and a sound image 
localization by processing stereophonic signals in- 
cluding a plurality of channel signals. The appara- 

25 tus includes: an input section for inputting the plu- 
rality of channel signals; a first signal processing 
section for receiving the plurality of channel sig- 
nals, for performing a filtering process after dividing 
each of the channel signals into a plurality of 

30 branched signals, and for outputting a plurality of 
first processed signals; a subtracting section for 
receiving at least two of the plurality of channel 
signals, for producing a difference signal by sub- 
tracting one of the two channel signals from the 

35 other channel signal, and for outputting the dif- 
ference signal; at least one pair of second signal 
processing sections, each for receiving the dif- 
ference signal, for delaying the difference signal by 
a predetermined time, for adjusting the level to a 

40 predetermined level, and for outputting a pair of 
second processed signals; at least one pair of 
adding sections for receiving the first processed 
signals and at least a pair of the second processed 
signals, for adding the first and the second pro- 

45 cessed signals at a predetermined ratio, and for 
outputting at least a pair of added signals; and at 
least one pair of reproducing sections, each for 
receiving a corresponding one of the added sig- 
nals, and for reproducing the corresponding signal 

50 at a predetermined position, wherein the sound 
image is localized by reproducing the first pro- 
cessed signals, and the sound fi Id is reproduced 
with presence by reproducing the second pro- 
cessed signals. 

55 In one embodiment of the invention, the pair of 

the second signal processing sections include: a 
first delay section for delaying both of the received 
pair of difference signals by a predetermined time 
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with respect to the first processed signals; a sec- 
ond delay section for delaying one of the pair of 
difference signals by a predeternnined time with 
respect to the other difference signal; and a nnul- 
tiplying section for multiplying the pair of difference 
signals by respective predetermined coefficients. 

In another embodiment of the invention, the 
predetermined coefficients, which are multiplied to 
th pair of difference signals, have reversed signs 
from each other, whereby one of the pair of dif- 
ference signals is an anti-phase signal of the other 
difference signal. 

In another embodiment of the invention, the 
predetermined delay time used in the second delay 
section is set based on a reach time difference 
between a pair of signals which' reach a listener 
from at least the pair of reproducing sections, 
whereby the listener simultaneously receives the 
signals from at least the pair of reproducing sec- 
tions. 

In another embodiment of the invention, the 
apparatus further includes a second adding section 
for receiving the pair of added signals and the two 
channel signals, for adding one of the pair of added 
signals to one of the two channel signals, and for 
adding the other added signals to the other channel 
signals. 

According to another aspect of the invention, 
the method is used for calculating filter coefficients 
for controlling sound field and sound image, based 
on a plurality of first impulse response signals and 
a pair of second impulse response signals, the 
plurality of first impulse response signals indicating 
impulse responses from loudspeakers reproducing 
audio signals to both ears of a listener, the pair of 
second impulse response signals indicating im- 
pulse responses from a reference loudspeaker at a 
position at which a sound image is localized to 
both ears of the listener. The method includes the 
steps of: (a) extracting features of the pair of sec- 
ond impulse response signals, and producing a 
parameter signals representing the features; (b) 
adjusting at least one of the plurality of first im- 
pulse response signals based on the parameter 
signals, and producing a pair of third impulse re- 
sponse signals having the same features as the 
extracted features; and (c) calculating the filter co- 
efficients for controlling the sound field and sound 
image, based on the plurality of first impulse re- 
sponse signals and the produced pair of third im- 
pulse response signals. 

In one embodiment of the invention, in step (c). 
th filter coefficients are set so that th pair of third 
impulse response signals ar substantially equal to 
a pair of fourth impulse response signals, the pair 
of fourth impulse response signals indicating a pair 
of impulse responses at both ears of the listener 
when impulse signals are reproduced from the 



reproducing loudspeakers. 

In another embodiment of the invention, the 
method further includes the steps of: (d) calculating 
a pair of impulse responses at both ears of the 

5 listener when the impulse signals are reproduced 
from the reproducing loudspeakers, based on the 
first impulse response signals and the filter coeffi- 
cients, and producing the pair of fourth impulse 
response signals; (e) comparing the pair of fourth 

70 impulse response signals with the pair of third 
impulse response signals, and producing a correla- 
tion signal; and (f) producing a control signal which 
controls the coefficient calculation, based on the 
correlation signal. In step (c). in accordance with 

75 the control signal, one of step (c1) of producing 
signals indicative of the calculated filter coefficients 
and step (c2) of calculating again the filter coeffi- 
cients using signals which are obtained by delaying 
the pair of third impulse response signals by a 

20 predetermined time. 

In another embodiment of the invention, step 

(a) includes the steps of: (a1) detecting a level ratio 
a of the pair of second impulse response signals, 
and producing a level ratio detection signal; and 

25 (a2) detecting a time difference dt of the pair of 
second impulse response signals, and producing a 
time difference detection signal. 

In another embodiment of the invention, step 

(b) includes the steps of: (b1) selecting one pair of 
30 first impulse response signals from among the plu- 
rality of first impulse response signals; (b2) adjust- 
ing the pair of first impulse response signals so 
that a relative time difference of the pair of first 
impulse response signals is equal to the time dif- 

35 ference dt based on the time difference detection 
signal, and producing a pair of adjusted impulse 
response signals; and (b3) adjusting a gain of the 
pair of the adjusted impulse signals so that the 
level ratio of the adjusted impulse response signals 

40 in the pair is equal to the level ratio a based on the 
level ratio detection signal, and producing the pair 
of gain-adjusted signals as the pair of third impulse 
response signals. 

In another embodiment of the invention, step 

45 (b) includes the steps of: (b4) selecting one first 
impulse response signal from among the plurality 
of first impulse response signals; (bS) delaying the 
selected first impulse response signal by the time 
difference dt based on the time difference detec- 

50 tion signal, and producing a delayed impulse re- 
sponse signal; and (b6) adjusting a gain of the 
delayed impulse response signal by multiplying the 
delayed impulse response signal by the level ratio 
a based on the level ratio detection signal, and 

55 producing an adjusted impulse response signal. 
The pair of third impulse response signals are 
constituted of the selected first impulse response 
signal and the adjusted impulse response signal. 
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In another embodiment of the invention, step 

(a) Includes the steps of {a3) detecting transfer 
characteristics of the pair of second impulse re- 
sponse signals, and (a4) calculating a transfer char- 
acteristic ratio, and producing a characteristic ratio 
signal. 

In another embodiment of the invention, step 

(b) includes the steps of: (b7) selecting one first 
impulse response signal from among the plurality 
of first impulse response signals; and (b8) adjusting 
a transfer characteristic of the selected first im- 
pulse response signal based on the characteristic 
ratio, and producing an adjusted impulse response 
signal. The pair of third impulse response signals 
ar constituted of the selected first impulse re- 
sponse signal and the adjusted impulse response 
signal. 

In another embodiment of the invention, step 
(a3) includes: a first transform step of transforming 
the received pair of second impulse response sig- 
nals into a pair of first characteristic signals repre- 
sented in frequency domain; and a first calculation 
step of calculating a transfer characteristic ratio of 
th pair of second impulse response signals based 
on the first characteristic signals, and step (bS) 
includes: a second transform step of transforming 
the selected first impulse response signal into a 
second characteristic signal represented in fre- 
quency domain; a second calculation step of mul- 
tiplying the second characteristic signal by the 
transfer characteristic ratio indicated by the char- 
acteristic ratio signal; and an inverse transform step 
of transforming the multiplied signal into a signal 
represented in time domain. 

In another embodiment of the invention, in the 
first and second transform steps. Fourier trans- 
forms are performed, and in the inverse transform 
st p, an Inverse Fourier transform is performed. 

According to another aspect of the invention, 
the sound field/sound image control method for 
performing a sound field control and a sound im- 
age localization by processing stereophonic signals 
including a plurality of channel signals, includes: an 
input step of inputting the plurality of channel sig- 
nals; a first signal processing step of performing a 
filtering process after dividing each of the channel 
signals into a plurality of branched signals, and 
producing a plurality of first processed signals; a 
subtracting step of subtracting one of at least two 
of th plurality of channel signals from the other 
channel signal, and producing a difference signal; a 
s cond signal processing step of delaying th dif- 
ference signal by a predetermined time, adjusting 
the lev I to a predetermined level, and producing a 
pair of second processed signals; an adding step 
of adding the first processed signals and at least a 
pair of the second processed signals at a predeter- 
mined ratio, and producing at least a pair of added 



signals; and a reproducing step of reproducing the 
pair of added signals at predetermined positions, 
wherein the sound image is localized by reproduc- 
ing the first processed signals, and the sound field 

5 is reproduced with presence by reproducing the 
second processed signals. 

In one embodiment of the invention, the sec- 
ond signal processing step includes: a first delay 
step of delaying both of the received pair of dif- 

10 ference signals by a predetermined time with re- 
spect to the first processed signals; a second delay 
step of delaying one of the pair of difference sig- 
nals by a predetermined time with respect to the 
other difference signal; and a multiplying step of 

75 multiplying the pair of difference signals by respec- 
tive predetermined coefficients. 

In another embodiment of the invention, the 
predetermined coefficients which are multiplied to 
the pair of difference signals have reversed signs 

20 from each other, whereby one of the pair of dif- 
ference signals is an anti-phase signal of the other 
difference signal. 

In another embodiment of the invention, the 
predetermined delay time used in the second delay 

25 step is set based on a reach time difference be- 
tween the pair of added signals reproduced in the 
reproducing step which reach a listener, whereby 
the listener simultaneously receives the reproduced 
pair of added signals. 

30 In another embodiment of the invention, the 
method further includes a second adding step of 
adding one of the pair of added signals to one of 
the two channel signals, and for adding the other 
added signals to the other channel signals. 

35 In this invention, impulse responses from a 

reference loudspeaker which are obtained by mea- 
surements or the like to respective ears of a lis- 
tener are not directly used as the reference char- 
acteristics for calculating filter coefficients. Instead, 

40 a pair of impulse responses from reproducing loud- 
speakers to the respective ears are used for the 
calculation. The relative time difference and the 
relative level (the level ratio) of the pair of impulse 
responses from the reproducing loudspeakers are 

45 controlled so as to be made equal to the time 
difference and the level ratio of a pair of impulse 
responses from the reference loudspeaker to the 
respective ears, thereby obtaining a pair of signals 
which are adopted. Accordingly, the difference in 

50 amplitude/frequency characteristics between the 
reference characteristics and the reproduction-sys- 
tem original characteristics can be minimized. Also, 
the relative time difference and th level difference 
between impulse responses at the respective ears 

55 of the listener during the sound image control are 
maintained in the reproduction-system original 
characteristics, so that it is possible to perform the 
sound image control with reduced deterioration of 
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sound quality. 

According to the invention, in the case where 
there are a plurality of listeners, for listeners on the 
center line in the arrangement of the reproducing 
loudspeakers, the expansion is realized by localiz- 
ing th L-ch and R-ch source signals in a region 
expanded from the located positions of the L-ch 
and R-ch reproducing loudspeakers. Also, for lis- 
teners at positions shifted from the center line, 
spatial expansion is realized by adjusting the delay 
amounts of the difference signals, including rever- 
beration components of the source signals and 
their anti-phase signals, so that the sounds from 
th respective reproducing loudspeakers simulta- 
neously reach the listeners. Accordingly, all the 
listeners positioned on the center line and at posi- 
tions shifted from the center line can feel expan- 
sion. Thus, it is possible to perform a sound field 
reproduction with presence in a wide service area. 

Thus, the invention described herein makes 
possible the advantage of providing a sound field 
and sound image control apparatus and a sound 
field and sound image control method with a re- 
duced deterioration in reproduced sound quality 
and with a wide service area. 

This and other advantages of the present in- 
V ntion will become apparent to those skilled in the 
art upon reading and understanding the following 
detailed description with reference to the accom- 
panying figures. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 schematically shows a method for 
localizing a sound image in the left rear of a 
listener by a sound field and sound image control 
apparatus in a first example according to the inven- 
tion. 

Figure 2 is a block diagram showing a sound 
image control coefficient calculating device for the 
sound field and sound image control of the first 
xample. 

Figure 3 shows an exemplary level ratio detec- 
tor. 

Figure 4 shows an exemplary time difference 
detector. 

Figure 5 schematically shows an exemplary 
time difference adjuster. 

Figure 6 schematically shows an exemplary 
lev I ratio adjuster. 

Figure 7 is a block diagram showing a sound 
image control coefficient calculating device in a 
s cond example according to the invention. 

Figur 8 schematically shows a method for 
localizing a sound image in the left rear of a 
listener by a sound field and sound image control 
apparatus in a third example according to the in- 
vention. 



Figure 9 is a block diagram showing a sound 
image control coefficient calculating device in the 
third example. 

Figure 10 is a block diagram of an exemplary 
5 transfer characteristic difference detector. 

Figure 11 is a block diagram of an exemplary 
transfer characteristic adjuster. 

Rgure 12 is a block diagram showing a sound 
field and sound image control apparatus in a fourth 
70 example according to the invention. 

Rgure 13 is a block diagram showing a sound 
field and sound image control apparatus in a fifth 
example according to the invention. 

Figure 14 schematically shows an exemplary 
75 construction of a conventional sound field and 
sound image control apparatus and a filter coeffi- 
cient calculating method for localizing the sound 
image in the left rear of a listener. 

Rgure 15 is a block diagram showing a basic 
20 construction of an FIR filter. 

Rgure 16 is a block diagram showing a con- 
ventional exemplary filter coefficient calculating de- 
vice for sound image localization. 

25 DESCRIPTION OF THE PREFERRED EMBODI- 
MENTS 

The present invention will be described by way 
of illustrative examples with reference to the ac- 
30 companying drawings. 

Example 1 

Rgure 1 schematically shows a method for 
35 localizing a sound image in the left rear of a 
listener 6 by a sound field and sound image control 
apparatus 100 in a first example according to the 
invention. 

In the apparatus 100. sound source signals 8- 
40 (n) generated by a sound source 1 are processed 
by FIR filters 2-1 and 2-2, and then the processed 
signals are reproduced from a L-ch reproducing 
loudspeaker 3 and a R-ch reproducing loudspeaker 
4. respectively. For the FIR filter 2-1. filter coeffi- 
45 dents H1(n) are set. For the FIR filter 2-2. filter 
coefficients H2(n) are set. In cases where the ap- 
paratus 100 is used for digital processing, an A/D 
converter and a D/A converter are required. For 
simplicity, such converters are omitted in the fig- 
so ure. The listener 6 stays at a position distant from 
the two loudspeakers 3 and 4 by equal distances 
(i.e.. on the center line), and faces th front (i.e.. 
faces toward the middle point between two loud- 
speakers). 

55 In Rgure 1, 01 (n) indicates an impulse re- 
sponse from the loudspeaker 3 at the position of 
the left ear of the listener 6 (to be more accurate, 
the position of the eardrum; and in the actual 
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measurement, it is measured at the entrance of the 
auditory canal when an impulse is Input to the L-ch 
loudspeaker 3). Similarly. C2(n) indicates an Im- 
pulse response from the L-ch loudspeaker 3 at the 
position of the right ear of the listener 6, C3(n) 
indicates an impulse response from the R-ch loud- 
speaker 4 at the position of the left ear of the 
listener 6. and C4(n) indicates an impulse response 
from the R-ch loudspeaker 4 at the position of the 
right ear of the listener 6. In addition. T1(n) and T2- 
(n) indicate impulse responses from a reference 
loudspeaker 5 to the left and right ears of the 
listener 6, respectively. The respective values of 
Cl(n) - C4(n), T1(n) and T2(n) can be obtained by 
actual measurements or simulation. 

In this example, the sound source signals S(n) 
are processed by the FIR filters 2-1 and 2-2 in the 
following manner. Rrst. a reach time difference dt 
and a level ratio a of a pair of signals respectively 
reaching the left and right ears of the listener 6 are 
obtained when the sound source signals S(n) are 
output from the reference loudspeaker 5 (the reach 
time difference dt and the level ratio a are param- 
eters indicative of the characteristics of reference 
Impulse responses). Then, the convolution process 
is performed in such a manner that a reach time 
difference and a level ratio of signals respectively 
reaching the left and right ears of the listener 6 
when the audio signals are output from the re- 
producing loudspeakers 3 and 4 are made equal to 
the reach time difference dt and the level ratio a. 

For example, when a pair of impulse responses 
from reproducing loudspeakers 3 and 4 to both 
ears of a listener are represented by L(n) (left ear) 
and R(n) (right ear), the relationship expressed by 
Equation (3) below must be established in order to 
satisfy the above condition, in this example, H1(n) 
and H2(n) which satisfy the condition of Equation 
(3) are set for the FIR filters 2-1 and 2-2. 

R(n) = a • L (n + r) (3) 

In the above equation, r indicates, when a 
signal S(n) Is output from the reference loud- 
sp ak r 5, a time difference dt in the notation of 
discrete time obtained by subtracting the time tp at 
which the signal reaches the right ear from the time 
tt at which the signal reaches the left ear; and a is 
obtained by dividing the level of the signal which 
reaches the right ear by the level of the signal 
which reaches the left ear. Usually, in the case 
where the loudspeaker 5 is located on the left side 
as is shown in Figure 1, t ^ 0. and a ^ 1. In 
addition, the time differenc dt and the level ratio a 
can be calculated by using the timings at which the 
peaks of the respective signals reach and the sig- 
nal levels at the peaks. 



Next, referring to Figure 2. a device and a 
method for calculating the filter coefficients (im- 
pulse responses) H1(n) and H2(n) in the sound 
field and sound image control apparatus 100 of this 

5 example will be described. Figure 2 is a block 
diagram showing a filter coefficient (hereinafter re- 
ferred to as sound Image control coefficient) cal- 
culating device 200 for the sound field and sound 
image control of this example. 

10 The device 200 includes reproduction-system 

characteristics input terminals 11-1 to 11-4 for in- 
putting signals representing impulse responses 
from two reproducing loudspeakers to both ears of 
a listener, and reference characteristics input termi- 

75 nals 12-1 and 12-2 for Inputting signals represent- 
ing impulse responses from the reference loud- 
speaker located at a position at which a sound 
image is to be localized to both ears of the listener. 
The impulse response signals which are Input to 

20 the respective input terminals correspond to the 
impulse responses C1(n) - C4(n) and the Impulse 
responses T1(n) and T2(n) shown in Figure 1. 
Hereinafter the impulse response signals corre- 
sponding to the respective impulse responses are 

25 represented by SCI (n). ST1 (n) and the like. 

The device 200 Includes a filter coefficient cal- 
culator 18, FIR filters 22-1. 22-2. and 23-1 to 23-4. 
a filter coefficient setting device 20, an Impulse 
generator 21, adders 24-1 and 24-2. correlation 

30 ratio calculators 25-1 and 25-2. a feedback control- 
ler 26. and filter coefficient output terminals 19-1 
and 19-2. The filter coefficient calculator 18 cal- 
culates a pair of filter coefficients (In the figure, 
indicated by H'1 (n) and H'2(n)) in accordance with 

35 the left sides of Equations (1 ) and (2). based on the 
impulse response signals SC1(n) to SC4(n) repre- 
senting the reproduction-system characteristics, 
and the pair of impulse response signals Sn(n) 
and ST'2(n) representing the reference characteris- 

40 tics. The filter coefficient setting device 20 sets the 
filter coefficients for the respective FIR filters 23-1 
to 23-4. 22-1 and 22-2. based on the impulse 
response signals SCI (n) to SC4(n) and the signals 
SH*1(n) and SH'2(n) representing the filter coeffi- 

45 cients which are all output from the filter coefficient 
calculator 18. The impulse generator 21 supplies 
an impulse signal S110 to the FIR filters 22-1 and 
22-2. The adders 24-1 and 24-2 add the signals 
S121 - S124 which are output from the FIR filters 

50 23-1 to 23-4. The correlation ratio calculators 25-1 
and 25-2 calculate correlation ratio of the outputs 
S130 and 8140 from the adders 24-1 and 24-2 and 
the Impulse response signals ST1(n) and ST'2(n). 
respectively. The feedback controller 26 compares 

55 the correlation ratios with a predetermined value, 
and controls the filter coefficient calculator 18 
based on the compared result The filter coefficient 
output terminals 19-1 and 19-2 output the final filter 
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coefficients H1(n) and H2(n) calculated by the filter 
coefficient calculator 18. 

The device 200 further includes a level ratio 
detector 13, a time difference detector 14, switches 
15-1 and 15-2. a time difference adjuster 16, and a 
level ratio adjuster 17. The level ratio detector 13 
detects a level ratio a of signal levels between the 
pair of impulse response signals ST1(n) and ST2- 
(n) input through the reference characteristics input 
terminals 12-1 and 12-2. The time difference detec- 
tor 14 detects a relative time difference dt between 
the pair of impulse response signals ST1(n) and 
ST2(n). The switches 15-1 and 15-2 select a pair of 
impulse response signals from among the impulse 
response signals SCl(n) - SC4(n) which are input 
through the reproduction-system characteristics in- 
put terminals 11-1 -11-4. The time difference ad- 
juster 16 adjusts a delay time so that the relative 
time difference between the pair of impulse re- 
sponse signals S101 and SI 02, which are selected 
by the switches 15-1 and 15-2, is made equal to 
the time difference dt The level ratio adjuster 17 
adjusts signal levels so that the level ratio of the 
pair of impulse response signals S105 and SI 06, 
which are output from the time difference adjuster 
16. is made equal to the level ratio a. The level 
ratio adjuster 17 outputs impulse response signals 
Sri(n) and Sr2(n) representing reference char- 
acteristics T'1(n) and r2(n). 

A method for calculating a sound image control 
coefficient performed by the sound image control 
coefficient calculating device 200 in the first exam- 
ple with the above-described construction will be 
d scribed below. 

Each of the impulse response signals SC1(n) - 
SC4(n), which are input through the reproduction- 
system characteristics input terminals 11-1 to 1-1-4, 
is branched into two signals which are in turn input 
to the filter coefficient calculator 18 and the switch 
15-1 or 15-2. respectively. The signals SC1(n) and 
SC3(n) are input to the switch 15-1, and the signal 
SC2(n) and SC4(n) are input to the switch 15-2. 
Each of the switches 15-1 and 15-2 selects one of 
the two input impulse response signals, and out- 
puts the selected signal to the time difference 
adjuster 16. At this stage, the pair of signals SC1(n) 
and SC2(n) are selected when the sound image is 
to be localized on the left side of the listener, and 
the pair of signals SC3(n) and SC4(n) are selected 
when the sound image is to be localized on the 
right side of the listener. The impulse response 
signals selected by the switches 15-1 and 15-2 ar 
input into the time difference adjuster 16 as signals 
3101 and SI 02, respectively. 

Each of th impulse response signals ST1(n) 
and ST2(n), which are input through the reference 
characteristics input terminals 12-1 and 12-2. is 
branched "into two signals, which are in turn input 



into the level ratio detector 13 and the time dif- 
ference detector 14. In the level ratio detector 13. 
the level ratio a of the signals ST1 (n) and ST2(n) is 
calculated, and the calculated level ratio is fed to 

5 the level ratio adjuster 17 as a level ratio detection 
signal S103. In the time difference detector 14, the 
relative time difference dt between the impulse 
response signals ST1(n) and ST2(n) is calculated, 
and the calculated time difference is output to the 

10 time difference adjuster 16 as a time difference 
detection signal S104. The time difference adjuster 
16 receives the pair of impulse response signals 
SI 01 and Si 02 from the switches 15-1 and 15-2 
and the time difference detection signal SI 04 from 

75 the time difference detector 14. Then, the time 
difference adjuster 16 adjusts the impulse response 
signals S101 and S102 so that the relative time 
difference between the impulse response signals 
SI 01 and S102 is made equal to the time dif- 

20 ference dt indicated by the time difference detec- 
tion signal Si 04. The adjusted signals are output to 
the level ratio adjuster 17 as the signals Si 05 and 
S106. 

The level ratio adjuster 17 receives the level 
25 ratio detection signal Si 03, the signals Si 05 and 
S106. and performs a gain adjustment so that the 
level ratio of the signals S105 and SI 06 is made 
equal to the level ratio a indicated by the level ratio 
detection signal S103. Then, the level ratio adjuster 
30 17 outputs a signal SI 07 (the reference character- 
istics signal Sri(n)) and a signal SI 08 (Sr2(n)) 
for calculating the filter coefficient to the filter co- 
efficient calculator 18. 

Rgure 3 shows an example of the level ratio 
35 detector 13 and a level ratio detecting method 
performed by the level ratio detector 13. For exam- 
ple, the level ratio detector 13 can be constructed 
by a divider 13-3, and peak detecting circuits 13-5 
and 13-6. Through input terminals 13-1 and 13-2, 
40 the impulse response signals ST1(n) and ST2(n) 
are input, respectively. By the peak detecting cir- 
cuits 13-5 and 13-6. a peak level A of the signal 
ST1 (n) and a peak level B of the signal ST2(n) are 
detected, respectively, and the detected values are 
45 fed to the divider 13-3. In the divider 13-3, a peak 
level ratio a = B/A is calculated and output from 
an output terminal 13-4 as the level ratio detection 
signal SI 03. In Figure 3 and also in Figures 4 to 6. 
the input signals ST1(n) and ST2(n) are schemati- 
se cally represented by showing the peak sound pres- 
sures A and B in which the horizontal axis denotes 
a time and the vertical axis denotes a voltage 
value. If the sound pressure is represented in deci- 
bel, a subtracter for calculating (A - B) is used 
55 instead of the divider. 

Rgure 4 shows an example of the time dif- 
ference detector 14 and a time difference detecting 
method performed by the time difference detector 
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14. The time difference detector 14 first detects 
times ti and t2 corresponding to the peak levels for 
the impulse response signals ST1(n) and ST2(n) 
which are input through input terminals 14-1 and 
14-2, respectively. The detecting circuits for detect- 
ing a peak of a signal level and for detecting a time 
corresponding to the peak can be realized by a 
conventional techniques using a microcomputer or 
the like. From the times ti and t2. a relative time 
difference dt is obtained and output through an 
output terminal 14-3 as the time difference detec- 
tion signal S104. 

Figure 5 schematically shows an example of 
the time difference adjuster 16 and a time dif- 
ference adjusting method performed by the time 
difference adjuster 16. The time difference adjuster 
16 first detects times t'l and t'2 corresponding to 
the peak levels of the impulse response signals 
SI 01 and SI 02 input through input terminals 16-1 
and 16-2, respectively. Herein, the pair of the sig- 
nals S101 and 8102 may be a pair of the impulse 
response signals SC1(n) and SC2(n). 

Through an input terminal 16-3, the time dif- 
ference detection signal 8104 is input. Based on 
the time difference dt indicated by the signal SI 04. 
the signal Si 02 is delayed so that the peak posi- 
tion of the signal 8102 is adjusted to be a time ta. 
That Is, the signal SI 02 Is delayed by t = tff - 1'2 
+ fi so that the time difference between t*i and ta 
Is made equal to dt The signal 8106 which is 
obtained by delaying the signal 8102 is output 
through an output terminal 16-5. The signal 8101 is 
directly output through an output terminal 16-4 as 
the output signal 8105. In this way, the time dlf- 
f rence at the peak sound pressure between the 
signals Si 05 and 8106 output from the time dif- 
ference adjuster 16 Is adjusted so as to be equal to 
the time difference dt indicated by the time dif- 
ference detection signal 8104. 

Rgure 6 Is a schematic diagram showing ah 
xample of the level ratio adjuster 17 and a level 
ratio adjusting method performed by the level ratio 
adjuster 17. The level ratio adjuster 17 can be 
constructed of peak detecting circuits 17-4 and 17- 
5, a multiplier 17-6, and a calculator 17-7 by using 
a conventional signal processing technique. 

Through an input terminal 17-1. the output sig- 
nal 8105 of the time difference adjuster 16. and 
through an input terminal 17-2, the signal 8106 is 
Input. By the peak detecting circuits 17-4 and 17-5. 
a peak sound pressure A' of the Input signal 81 05 
and a peak sound pressur B* of the input signal 
8106 are detected, respectively. 

Through an input terminal 17-3. the level ratio 
detection signal 8103 Is Input from the lev I ratio 
detector 13. The calculator 17-7 receives signals 
Indicating the peak sound pressures A* and B' and 
the signal 8103 indicating the level ratio a. and 



calculates (A'« a) / B'. The calculated result is 
output to the multiplier 17-6. The multiplier 17-6 
multiplies the input signal 8106 by the calculated 
result (A* -a) / B*. and the resulting signal 8108 is 

5 output. The peak level of the output signal S108 is 
A' •a, so that the level ratio of the signals 8108 and 
8105 is a. The output signal having the peak level 
A'«a is output through an output terminal 17-9 as 
an impulse response signal 8T'2(n). The signal 

70 8105 is directly output through an output terminal 
17-8 as the output signal 8107. In this way. the 
signals 8107 and 8108 output from the level ratio 
adjuster 17 have a peak ratio which is equal to the 
peak ratio a which is given by the peak ratio 

15 detection signal 8103. These signals 8107 and 
Si 08 are fed to the filter coefficient calculator 18 
as the impulse response signals ST'1(n) and 8r2- 
(n). respectively. 

The filter coefficient calculator 18 receives the 

20 impulse response signals SC1(n) - SC4(n) applied 
through the reproduction-system characteristics in- 
put terminals 11-1 - 11-4. and also receives the 
impulse response signals Sri(n) and 8r2(n) ap- 
plied from the level ratio adjuster 17. The filter 

25 coefficient calculator 18 calculates filter coefficients 
HM(n) and H*2(n) which satisfy Equations (4) and 
(5) below, based on the impulse responses C1(n) - 
C4(n), ri(n) and r2(n). 

30 ri(n) = H'1(nrC1(n) + H*2(nrC3(n) (4) 

r2(n) = H'1(n)*C2(n) + H'2(nrC4(n) (5) 

The filter coefficient calculator 18 can be con- 
structed as a matrix calculator. Instead of the ma- 

35 trix calculator, it is possible to use another calcula- 
tor in which the coefficients are obtained by per- 
forming the Fourier transform for the impulse re- 
sponse, and performing the operation in the fre- 
quency domain. 

40 The impulse response signals SC1(n) - SC4(n) 
and the impulse response signals SH'1(n) and 
SH*2(n) based on the calculated results* are fed to 
the filter coefficient setting device 20. The filter 
coefficient setting device 20 sets the coefficient 

45 H'1 (n) for the FIR filter 22-1 and the coefficient H'2- 
(n) for the FIR filter 22-2, as their tap coefficients. 
Similarly, for the FIR filters 23-1 - 23-4, the impulse 
responses C1(n) - C4(n) are set. 

After the tap coefficients of the FIR filters are 

50 set. a pulse signal 8110 is supplied from the im- 
pulse generator 21 to the FIR filters 22-1 and 22-2. 
The filters 22-1 and 22-2 perform the filtering pro- 
cesses (convolution) in accordanc with their tap 
CO fficients (impulse responses H*1(n) and H*2(n)). 

55 The resulting signal 8111 Is branched into two, 
signals which are in turn input to the FIR filters 23- 
1 and 23-2. The resulting signal 8112 is branched 
into two signals which are in turn input to the FIR 
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filters 23-3 and 23-4. The FIR filters 23-1 - 23-4 
perform the filtering processes in accordance with 
their tap coefficients (impulse responses Cl(n) - 
C4(n)), and outputs resulting signals S121 - Si 24. 

The adder 24-1 receives the signals S121 and 
SI 23, and adds the signals to each other. The 
resulting added signal 8130 is supplied to the 
correlation ratio calculator 25-1. The adder 24-2 
receives the signals Si 22 and SI 24, and adds the 
signals to each other. The resulting added signal 
SI 40 is supplied to the correlation ratio calculator 
25-2. 

The added signal SI 30 corresponds to the 
calculation result shown in the right side of Equa- 
tion (4). and the added signal S140 corresponds to 
th calculation result shown in the right side of 
Equation (5). That is. the added signals S130 and 
SI 40 correspond to the impulse responses L(n) 
and R(n) which are realized at the left-ear and 
right-ear positions of a listener by the calculated 
filt r coefficients H'1(n) and H'2(n). 

The correlation ratio calculator 25-1 calculates 
a correlation ratio of the impulse response n(n) 
which is applied from the level ratio adjuster 17 as 
the reference characteristics to the added signal 
SI 30 applied from the adder 24-1, thereby generat- 
ing a correlation ratio signal S131. Similarly, the 
correlation ratio calculator 25-2 calculates a correla- 
tion ratio of the impulse response T'2(n) which is 
applied from the level ratio adjuster 17 as the 
reference characteristics to the added signal 8140 
applied from the adder 24-2. thereby generating a 
correlation ratio signal SI 41. Each of the correla- 
tion ratio calculators 25-1 and 25-2 can be con- 
structed of a subtracter and an adder (and. if 
necessary, a divider for dividing the subtracted 
result by the added result) by using a conventional 
technique. For example, the subtracter may sub- 
tract one of two input signals from the other and 
output an absolute value of the obtained difference, 
and the adder may add the respective absolute 
values of two input signals to each other. In the 
case where the divider is used, the correlation ratio 
can be a value of 0 to 1 . 

The feedback controller 26 receives the cor- 
relation ratio signals SI 31 and SI 41. and compares 
the signals with a predetermined value. Based on 
th compared result, the feedback controller 26 
generates a control signal S150 which is supplied 
to the filter coefficient calculator 18. If the correla- 
tion ratios indicated by the correlation ratio signals 
SI 31 and Si 41 are equal to or larger than the 
predetermined value, the control signal 8150 in- 
structs th filter co fficient calculator 18 to stop the 
operation. Otherwise, the control signal SI 50 in- 
structs the calculator 18 to continue the operation. 

The filter coefficient calculator 18 stops the 
filter coefficient calculation if the stop is instructed 



by the control signal SI 50 applied from the feed- 
back controller 26. In this case, the filter coefficient 
calculator 18 outputs the filter coefficients H'1(n) 
and H*2(n). which have been obtained in the pre- 

5 vious calculation, through filter coefficient output 
terminals 19-1 and 19-2 as the final filter coeffi- 
cients H1(n) and H2(n). In the case where the 
calculation is instructed to be continued by the 
control signal SI 50, the impulse responses T'1(n) 

70 and T*2(n) are delayed by a predetermined time, 
and again the filter coefficients H'1(n) and H*2(n) 
are calculated. Then, the same processes are re- 
peated. 

The feedback control is performed for com- 

75 pensating the delay due to the filtering processes 
in the FIR filters 22-1 and 22-2. and can be per- 
formed by a software processing using a dedicated 
microcomputer. As a result of the feedback control, 
the right sides of Equations (4) and (5) can be used 

20 for calculating the filter coefficients H1(n) and H2(n) 
which are more accurately in accord with not only 
the profiles of the impulse responses T*1(n) and 
T'2(n) but also the times of the impulse responses. 
In this way. in the case, for example, where the 

25 sound image is to be localized on the left side of 
the listener 6 by the sound field and sound image 
control apparatus 100, it is possible to minimize the 
difference between the sound quality of the sound 
image localized by the apparatus 100 and the 

30 sound quality of the sound reproduced from the 
left-side (the side on which the sound image is 
localized) reproducing loudspeaker 3 without using 
the apparatus 100. Similarly, in the case where the 
sound image is to be localized on the right side of 

35 the listener 6 by the apparatus 100, it is possible to 
minimize the difference between the sound quality 
of the localized sound image and the sound quality 
of the sound reproduced from the right-side re- 
producing loudspeaker 4 without using the appara- 

40 tus 100. 

In this example, the cases where the sound 
image is to be localized on the left side and the 
right side of the listener 6 are described. Alter- 
natively, irrespective of the position at which the 

45 sound image is to be localized, either a pair of C1- 
(n) and C2(n) or a pair of C3(n) and C4(n) may be 
used. 

As described above, the device 200 in this 
example does not directly use the impulse re- 

50 sponses T1(n) and T2(n) from the reference loud- 
speaker 5 actually located at a position at which 
the sound image is localized to both ears of the 
listener 6. The device 200 in this example uses, as 
the reference characteristics, the impulse re- 

55 sponses TM(n) and T*2{n) which are obtained by 
controlling the level ratio and the relative time 
difference of the (pair of) impulse responses from 
one of the reproducing loudspeakers 3 and 4 to 
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both ears of the listener 6. thereby calculating the 
filter coefficients. Accordingly, it is possible to re- 
duce the change in sound quality of the localized 
sound image while maintaining the effects of the 
sound image localization. 

Also, as described above, the filter coefficients 
for sound image control are calculated while the 
impulse responses ri(n) and T'2(n) representing 
the reference characteristics are both delayed by a 
very little time period using a method of successive 
approximation (iteration method), whereby more 
accurate results can be obtained. 

Example 2 

Next, a device for calculating-sound image 
control coefficients and a sound image control co- 
efficient calculating method in a second example 
according to the invention will be described. Figure 
7 is a block diagram showing a sound image 
control coefficient calculating device 300 of the 
second example. 

The device 300 includes reproduction-system 
characteristics input terminals 11-1 - 11-4, refer- 
ence characteristics input terminals 12-1 and 12-2, 
a filter coefficient calculator 18, FIR filters 22-1, 22- 
2. and 23-1 - 23-4, a filter coefficient setting device 
20, an impulse generator 21, adders 24-1 and 24-2, 
a correlation ratio calculators 25-1 and 25-2, a 
fe dback controller 26, and filter coefficient output 
terminals 19-1 and 19-2. These components and 
elements are the same as those used in the device 
200 in the first example, so that the descriptions 
ther of are omitted. 

The device 300 further includes a level ratio 
detector 13, a time difference detector 14. a switch 
31, a time difference adjuster 32, and a level ratio 
adjuster 33. Among them, the level ratio detector 
13 and the time difference detector 14 are the 
same as those in the device 200 in the first exam- 
ple. 

Each of the impulse response signals SCI (n) - 
SC4(n) input through the reproduction-system char- 
acteristics input terminals 11-1 - 11-4 is branched 
into two signals, which are in turn input into the 
filter coefficient calculator 18 and the switch 31. 
The switch 31 selects one of the four input impulse 
response signals and output the selected signal. 
The selected impulse response signal S201 is 
branched into two signals, which are in turn applied 
to the time difference adjuster 32 and the filter 
coefficient calculator 18. Th impulse response sig- 
nal S201 applied to th filter coefficient calculator 
18 is directly used as the reference characteristic 
n(n) for calculating the filter coefficients. 

Each of the impulse response signals ST1(n) 
and ST2(n) input through the reference characteris- 
tics input terminals 12-1 and 12-2 is branched into 



two signals, which are in turn input to the level ratio 
detector 13 and the time difference detector 14. In 
the level ratio detector 13, a level ratio a of the 
signals ST1(n) and ST2(n) is calculated, and the 

5 calculated result is applied to the level ratio ad- 
juster 33 as a level ratio detection signal S103. In 
the time difference detector 14, a relative time 
difference dt between the impulse response sig- 
nals ST1(n) and ST2(n) is calculated, and the cal- 

10 culated result is output to the time difference ad- 
juster 32 as a time difference detection signal 
SI 04. The constructions and the operations of the 
level ratio detector 13 and the time difference de- 
tector 14 are the same as those in the device 200 

75 described in the first example. 

The time difference adjuster 32 receives the 
impulse response signal S201 output from the 
switch 31 and the time difference detection signal 
Si 04 output from the time difference detector 14. 

20 The time difference adjuster 32 delays the impulse 
response signal S201 by a time corresponding to 
the time difference dt indicated by the time dif- 
ference detection signal S104. The delayed signal 
is output to the level ratio adjuster 33 as a signal 

25 S205. 

The level ratio adjuster 33 receives the signal 
S205 and the level ratio detection signal S103, and 
performs the gain adjustment by multiplying the 
delayed impulse response signal S205 by the level 

30 ratio a indicated by the level ratio detection signal 
S103. Then, the gain-adjusted signal 8208 is out- 
put to the filter coefficient calculator 18. The signal 
S208 is a signal obtained by delaying the impulse 
response signal S201 (i.e., the reference character- 

35 istics signal ST'1(n)) by a time dt and by multiply- 
ing the level by a. The signal 8208 is input to the 
filter coefficient calculator 18 as the other reference 
characteristics signal 8r2(n) for calculating the fil- 
ter coefficients. 

40 The filter coefficient calculator 18 receives the 
impulse response signals SCl(n) - SC4(n) applied 
through the reproduction-system characteristics in- 
put terminals 11-1 - 11-4. the impulse response 
signal 8201 (i.e.. the reference characteristics sig- 

45 nal Sn(n)) applied from the switch 31, and the 
impulse response signal 8208 (i.e., ST*2(n)) applied 
from the level ratio adjuster 33. Based on the 
impulse responses C1(n) - C4(n), T1(n), and T'2- 
(n), the filter coefficient calculator 18 calculates the 

50 filter coefficients H*1(n) and H'2(n) which satisfy 
Equations (4) and (5) above, the same as in the 
device 200. 

The subsequent signal processes are the same 
as those in the device 200 described in the first 
55 exampi , and the final filter coefficients H1(n) and 
H2(n) are output through the output terminals 19-1 
and 19-2. 
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As described above, the device 300 in this 
example does not directly use the innpulse re- 
sponses T1(n) and T2(n) from the reference loud- 
speaker 5 actually located at a position at which 
the sound image is to be localized to both ears of 
the listener 6. The device 300 In this example uses, 
as the reference characteristics, an impulse re- 
sponse (T'1(n)) from one of the reproducing loud- 
speakers to one of the ears of the listener 6, and 
an impulse response (T'2(n)) which is obtained by 
controlling the level ratio and the relative time 
difference of the impulse response, thereby cal- 
culating the filter coefficients. Accordingly, it is 
possible to reduce the change in sound quality of 
the localized sound image while maintaining the 
ffects of the sound image localization. 

Example 3 

Next, a sound field and sound image control 
apparatus, and a device and a method for calculat- 
ing sound image control coefficients in a third 
example according to the invention will be de- 
scribed. 

Rgure 8 schematically shows a method for 
localizing a sound image in the left rear of a 
list ner 6 by a sound field and sound image control 
apparatus 400 in the third example. 

In the apparatus 400, sound source signals S- 
(n) generated by a sound source 1 are processed 
by FIR filters 2-3 and 2-4, and then the processed 
signals are reproduced from a L-ch reproducing 
loudspeaker 3 and a R-ch reproducing loudspeaker 
4, respectively. For the FIR filter 2-3, filter coeffi- 
cients H1(n) are set. For the FIR filter 2-4. filter 
coefficients H2(n) are set. In cases where the ap- 
paratus 400 is used for digital processing, an A/D 
converter and a D/A converter are required. For 
simplicity, such converters are omitted in the fig- 
ur . The listener 8 stays at a position distant from 
the two loudspeakers 3 and 4 by equal distances 
(I. on the center line), and faces the front (i.e.. 
faces toward the middle point between two loud- 
sp akers). The construction of the apparatus 400 is 
the same as that of the apparatus 100 described in 
the first example, except for the constructions and 
the operations of the FIR filters 2-3 and 2-4. 

In this example, the audio signals are pro- 
cessed by the FIR filters 2-3 and 2-4 in such a 
manner that the impulse responses at a position of 
a first-side ear (i.e.. the ear closer to a sound 
image to be localized) when the audio signals after 
the convolution process by the FIR filters 2-3 and 
2-4 are output from the reproducing loudspeakers 3 
and 4 so as to localize a sound image on the first 
side (left or right) of th listener 6 are made equal 
to the impulse responses at th position of the first- 
side ear when the sound source signals are directly 



output from the loudspeaker located on the first 
side of the listener 6 without any process. 

Also, the FIR filters 2-3 and 2-4 perform the 
convolution processes so that the difference in 

5 transfer characteristics between the ears of the 
listener 6 when the signals obtained by processing 
the signals S(n) by the FIR filters 2-3 and 2-4 are 
output from the reproducing loudspeakers 3 and 4 
is made equal to the difference in transfer char- 

70 acteristics between the ears of the listener 6 when 
the signals S(n) are output from the reference 
loudspeaker 5. 

As in the first example, in Figure 8. Cl(n) 
indicates an impulse response from the loud- 

75 speaker 3 at the position of the left ear of the 
listener 6. Similarly. C2(n) indicates an impulse 
response from the L-ch loudspeaker 3 at the posi- 
tion of the right ear of the listener 6. C3(n) in- 
dicates an impulse response from the R-ch loud- 

20 speaker 4 at the position of the left ear of the 
listener 6, and C4(n) indicates an impulse response 
from the R-ch loudspeaker 4 at the position of the 
right ear of the listener 6. In addition, T1(n) and T2- 
(n) indicate impulse responses from the reference 

25 loudspeaker 5 to the left and right ears of the 
listener 6. respectively. The respective values of 
Cl(n) - C4(n), T1(n) and T2(n) can be obtained by 
actual measurements or simulation. In addition, a 
pair of impulse responses from the loudspeakers 3 

30 and 4 to both ears of the listener 6 when the audio 
signals processed by the FIR filters 2-3 and 2-4 are 
reproduced from the loudspeakers 3 and 4 are 
represented by L(n) (the left ear) and R(n) (the right 
ear). 

35 For example, in order to satisfy the above two 

conditions when the sound image is to be localized 
on the left side of the listener 6. the conditions 
expressed by Equations (6) and (7) below should 
be established. 

40 

L(n) = Cl(n) (6) 
F[L(n)] / F[R(n)] = F[T1 (n)] / F[T2(n)] (7) 

In the equations, F[ ] denotes a Fourier trans- 
45 form, that is, a transform from a time domain to a 
frequency domain. 

The impulse response R(n) is obtained on the 
basis of Equations (6) and (7) as follows: 

50 R(n) = F-^{F[C1 (n)]- F[T2(n)] / F[T1 (n)]} (8) 

In the above equation, F~i{ } denotes an in- 
V rse Fourier transform, that is, a transform from a 
frequency domain to a tim domain. 
55 The impulse responses L(n) and R(n) satisfy 
the following conditions expressed by Equations (9) 
and (10) below. 
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L(n) = H1(nrCl(n) + H2(n)"C3(n) (9) 
R(n) = H1(nrC2(n) + H2(nrC4(n) (10) 

On the basis of Equations (6) and (8) through (10), 
the following is obtained: 

C1(n) = H1(n)*C1(n) + H2(n)-C3(n) (11) 
F-HF[C1(n)].F[T2(n)]/F[T1(n)]} 
= H1(n)*C2(n) + H2(n)'C4(n) (12) 

In this example, for the FIR filters 2-3 and 2-4. 
the coefficients H1(n) and H2(n) which satisfy the 
conditions of Equations (11) and (12) are set. 

Next, referring to Rgure 9. a device and a 
method for calculating the filter coefficients (inn- 
pulse responses) H1(n) and H2(n)"-in the sound 
field and sound image control apparatus 400 of the 
third example will be described. Figure 9 is a block 
diagram showing a sound image control coefficient 
calculating device 500 in the third example. 

Similar to the devices 200 and 300, which are 
described in the first and second examples, the 
device 500 includes reproduction-system charac- 
teristics input terminals 11-1 - 11-4, reference char- 
acteristics input terminals 12-1 and 12-2, a filter 
coefficient calculator 18, FIR filters 22-1. 22-2, and 
23-1 - 23-4, a filter coefficient setting device 20, an 
impulse generator 21, adders 24-1 and 24-2, cor- 
relation ratio calculators 25-1 and 25-2, a feedback 
controller 26. and filter coefficient output terminals 
19-1 and 19-2. These components are the same as 
those in the devices 200 and 300. so that the 
descriptions thereof are omitted. 

The device 500 further includes a transfer char- 
acteristic difference detector 41, a transfer char- 
acteristic adjuster 42, and a switch 31. The switch 
31 is the same as that In the device 300. 

Each of the impulse response signals SC1(n) - 
SC4(n) input through the reproduction-system char- 
acteristics input terminals 11-1 - 11-4 is branched 
into two signals which are in turn input to the filter 
coefficient calculator 18 and the switch 31. The 
switch 31 selects one of the four input impulse 
response signals and outputs the selected one. 
The selected impulse response signal S201 is 
branched into two signals which are applied to the 
transfer characteristic adjuster 42 and the filter 
coefficient calculator 18. The impulse response sig- 
nal S201 . applied to the filter coefficient calculator 
18. is directly used as the reference characteristic 
T*1(n) for calculating the filter coefficients. 

The impulse response signals ST1 (n) and ST2- 
(n) input through the reference characteristics input 
terminals 12-1 and 12-2 are input into the transfer 
characteristic difference detector 41. In the transfer 
characteristic differenc detector 41. the transfer 
characteristics of both of the signals ST1(n) and 
ST2(n) are calculated, and a ratio of transfer char- 



acteristic at each frequency is detected. Specifi- 
cally, the transfer characteristic ratio on the fre- 
quency axis is calculated in accordance with the 
right side of Equation (7) above. The calculated 

5 ratio is output to the transfer characteristic adjuster 
42 as a detection signal S301. 

The transfer characteristic adjuster 42 performs 
the operation shown in the left side of Equation 
(12). based on the impulse response signal S201 

10 applied from the switch 31 and the detection signal 

5301. The obtained result is output as a signal 

5302. The signal S302 is applied to the filter coeffi- 
cient calculator 18, and used as the reference 
characteristic T'2(n) for calculating the filter coeffi- 

T5 cients. 

Rgure 10 is a block diagram of an example of 
the transfer characteristic difference detector 41 
and a method for detecting the transfer characteris- 
tic ratio performed by the transfer characteristic 
20 difference detector 41. The transfer characteristic 
difference detector 41 can be constructed of 
Fourier transformers 41-3 and 41-4, and a divider 

41- 5. These circuits can be realized by a conven- 
tional technique using a microcomputer or the like. 

25 The impulse response signals ST1(n) and ST2- 
(n), input through input terminals 41-1 and 41-2, 
are first processed (Fourier transformed) by the 
Fourier transformers 41-3 and 41-4. respectively. 
The Fourier transformer 41-3 outputs a signal F[T1- 

30 (n)] in the frequency domain to the divider 41-5. 
The Fourier transformer 41-4 outputs a signal, F- 
[T2{n)] in the frequency domain to the divider 41-5. 
In the divider 41-5. the transfer characteristic ratio 
F[T2(n)] / F[T1(n)] is calculated, and the result is 

35 output from an output terminal 41-6 as the signal 
S301. 

Rgure 11 is a block diagram of an example of 
the transfer characteristic adjuster 42, and a meth- 
od for adjusting the transfer characteristic per- 
40 formed by the transfer characteristic adjuster 42. 
The transfer characteristic adjuster 42 can be con- 
structed of a Fourier transformer 42-3. a multiplier 

42- 4, and an inverse Fourier transformer 42-5. 
These circuits can be realized by a conventional 

45 technique using a microcomputer or the like. 

The impulse response signal 8201, (Ci(n); i is 
one of 1 - 4) input through an input terminal 42-1, 
is processed (Fourier transformed) by the Fourier 
transformer 42-3. and then output to the multiplier 

50 42-4 as a signal F[Ci(n)] on the frequency axis. The 
multiplier 42-4 multiplies the signal F[Ci(n)] by the 
transfer characteristic ratio F[T2(n)] / F[T1(n)] 
based on the signal S301 input through an input 
terminal 42-2. The multiplication result F[Ci(n)] • F- 

55 [T2(n)] / F[T1(n)] is output to the inverse Fourier 
transformer 42-5. The inverse Fourier transformer 
42-5 transforms the multiplication result into an 
impulse response signal F-^{F[Ci(n)] • F[T2(n)] / 

HO 
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F[T1(n)]} on a time axis. The resulting impulse 
?esponse signal is output through an output termi- 
nal 42-6 as the signal S302. 

The impulse response signal S302 output from 
the transfer characteristic adjuster 42 is input to the 
filter coefficient calculator 18 as the other reference 
characteristics signal Sr2(n) for the filter coeffi- 
cient calculation. 

The filter coefficient calculator 18 receives the 
impulse response signals SC1(n) - SC4(n) applied 
through the reproduction-system characteristics in- 
put terminals 11-1 - 11-4, the impulse response 
signal S201 (i.e.. the reference characteristics sig- 
nal Sri(n)) applied from the switch 31, and the 
impulse response signal S302 (i.e., Sr2(n)) applied 
from the transfer characteristic adjuster 42. Based 
on the impulse responses C1(n)-C4(n), T'1(n), and 
r2(n), the filter coefficients H*1(n) and H*2(n) which 
satisfy the conditions of Equations (11) and (12) are 
calculated, similar to the devices 200 and 300. 

The subsequent signal processes are the same 
as those in the devices 200 and 300 described in 
th first and second examples, and the filter coeffi- 
cients H1(n) and H2(n) are finally output through 
the output terminals 19-1 and 19-2. 

As described above, the sound image is lo- 
calized on the left side of the listener 6 by realizing 
th transfer characteristic ratio of impulse response 
b tween the left and the right ears of the listener 6 
(th difference between transfer characteristics of 
head-related transfer functions) when the sound 
source is located on the left side, with the two 
reproducing loudspeakers 3 and 4. At the same 
time, the impulse response from the localized 
sound image to the left ear of the listener 6 is 
made equal to the impulse response from the L-ch 
loudspeaker 3 in front of the listener 6 to the left 
ear of the listener 6. whereby the change in sound 
quality of the sound image can be minimized. 

In the above example, the sound image is 
localized on the left side of the listener 6. If the 
sound image is to be localized on the right side of 
the listener 6. the coefficients H1(n) and H2(n) can 
be set so as to satisfy the conditions of Equations 
(13) and (14) below. 

C4(n) = H1(n)*C2(n) + H2(n)*C4(n) (13) 
F-^{F[C4(n)] . FtT1(n)] / F[T2(n)]} 

= H1(n)-C1(n) + H2(n)*C3(n) (14) 

As described above, the device 500 in this 
example does not directly use the impulse re- 
sponses T1(n) and T2(n) from th reference loud- 
speaker 5 actually located at a position at which 
th sound imag is to be localized to both ears of 
th listener 6. The device 500 in this example uses, 
as the reference characteristics, an impuls re- 
sponse {T'1(n)) from one of the reproducing loud- 



speakers to one of the ears of the listener 6, and 
an impulse response (T*2(n)) which is obtained by 
controlling the transfer characteristic of the impulse 
response, thereby calculating the filter coefficients. 
5 Accordingly, it is possible to reduce the change in 
sound quality of the localized sound image while 
maintaining the effects of the sound image localiza- 
tion. 

In the first to third examples, cases where the 
70 sound image is localized on either side of the 
listener 6 have been described. Alternatively, if the 
sound image is to be localized at the rear of the 
listener 6. the constructions and the processes are 
the same as In the above cases. In an alternative 
75 case where a so-called surround signal is localized 
on the side of the listener 6 and a main signal Is 
localized fonwardly, the sound quality of the sur- 
round signal can be made equal to the sound 
quality of the main signal, by using the apparatus 
20 of the invention described in the first to third exam- 
ples. Thus, it is possible to realize the sound field 
and sound image reproduction with natural expan- 
sion and presence. 

25 Example 4 

Next, a sound field and sound image control 
apparatus, and a sound image control method ac- 
cording to a fourth example of the invention will be 

30 described. In this example, an apparatus which can 
provide a plurality of listeners with expansion and 
presence is described. 

Figure 12 is a block diagram showing the 
sound field and sound Image control apparatus 600 

35 in the fourth example. 

The apparatus 600 Includes stereo signal input 
terminals 51-1 and 51-2. a subtracter 52. delay 
elements 53-1 - 53-6, multipliers 54-1 - 54-4. FIR 
filters 55-1 - 55-4, adders 56-1 and 56-2. and 

40 reproducing loudspeakers 57-1 and 57-2. Through 
the stereo signal input terminals 51-1 and 51-2. 
stereo signals SL(n) and SR(n) are input. The sub- 
tracter 52 calculates a difference between the ste- 
reo signals SL(n) and SR(n). so as to obtain a 

45 difference signal D(n). Each of the delay elements 
53-1 - 53-6 receives a corresponding branched 
difference signal D(n), and delays the signal by a 
predetermined time. The times delayed by the 
delay elements 53-1 - 53-6 are respectively pre- 

50 determined. The multipliers 54-1 - 54-4 perform the 
gain adjustment by multiplying the delayed dif- 
ference signals D(n) by respective predetermined 
coefficients (g1 - g4). The FIR filters 55-1 - 55-4 
perform the filtering process to th stereo signals 

55 SL(n) and SR(n) (the filter coefficients H1(n) - H4- 
(n)). The adders 56-1 and 56-2 add the signals 
output from the FIR filters 55-1 - 55-4 and the 
signals output from the multipliers 54-1 - 54-4. The 
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reproducing loudspeakers 57-1 and 57-2 reproduce 
the output signals from the adders 56-1 and 56-2. 
A first listener 58-1 stays at a center position in 
front of the two reproducing loudspeakers 57-1 and 
57-2. A second listener 58-2 stays on the left side 
of the first listener 58-1. A third listener 58-3 stays 
on the right side of the first listener 58-1. Herein, 
the coefficients g1 - g4 used in the multipliers 54-1 
- 54-4 are not limited to positive values. For exam- 
ple, the coefficients g1 and g2 in the multipliers 54- 

1 and 54-2 for the signals to be reproduced from 
the L-ch loudspeaker 57-1 may be set so as to be 
positive values, and the coefficient g3 and g4 in the 
multipliers 54-3 and 54-4 for the signals to be 
reproduced from the R-ch loudspeaker 57-2 may 
be set so as to be negative values. In such a 
s tting, more increased presence can be expected. 

The operation of the apparatus 600 with the 
above construction is now described. 

The stereo signal SL(n), input through the ste- 
reo signal input terminal 51-1, is branched into two 
signals, one of which is input to the subtracter 52. 
The other signal is further branched into two sig- 
nals which are input to the FIR filters 55-1 and 55- 
2. Similarly, the stereo signal SR(n), input through 
the stereo signal input terminal 51-2. is branched 
into two signals, one of which is input to the 
subtracter 52. The other signal is further branched 
into two signals which are input to the FIR filters 
55-3 and 55-4. The signals which flow from the 
stereo signal input terminals 51-1 and 51-2 to the 
FIR filters 55-1 - 55-4 are referred to as signals in a 
first system. 

The FIR filters 55-1 - 55-4 perform the filtering 
process to the input signals with their filter coeffi- 
cients H1(n) - H4(n). The processed results from 
the F!R filters 55-1 and 55-3 are output to the 
adder 56-1. and the processed results from the FIR 
filters 55-2 and 55-4 are output to the adder 56-2. 

Herein, the filter coefficients H1(n) and H2(n) 
ar set so that the sound image of the signal SL(n) 
is localized at an expanded position to the left from 
the position of the L-ch reproducing loudspeaker 
57-1 with respect to the first listener 58-1 who 
stays at the center front position, when the L-ch 
signal SL(n) is input through the stereo signal input 
terminal 51-1 and reproduced from the reproducing 
loudspeakers 57-1 and 57-2. Also, the filter coeffi- 
ci nts H3(n) and H4(n) are set so that the sound 
imag of the signal SR(n) is localized at an ex- 
panded position to the right from the position of the 
R-ch reproducing loudspeaker 57-2 with respect to 
the first listener 58-1, when the R-ch signal SR(n) 
is input through the stereo signal input terminal 51- 

2 and reproduced from the reproducing loud- 
speakers 57-1 and 57-2. The method for localizing 
th sound image of the signal SL{n) on the left side 
of the listener by using the FIR filters 55-1 and 55- 



2 (H1(n) and H2(n)), and the method for localizing 
the sound image of the signal SR(n) on the right 
side of the listener by using the FIR filters 55-3 and 
55-4 {H3(n) and H4(n)) are the same as those used 

5 in the conventional technique. 

In this way. the sound image control is per- 
formed by using the first-system signals, and the 
sound images are localized at the expanded posi- 
tions from the respective reproducing loud- 

10 speakers, so that the first listener 58-1 at the center 
front position can feel greater expansion as com- 
pared with the conventional stereo reproduction. 

On the other hand, the stereo signals SL(n) and 
SR(n), which are input through the stereo signal 

75 input terminals 51-1 and 51-2 and applied to the 
subtracter 52. are processed by subtraction in the 
subtracter 52. The subtracter 52 outputs the dif- 
ference signal D(n) (= SL(n) - SR{n)). The dif- 
ference signal D(n) is a signal including reverbera- 

20 tion components of the input stereo signals (some- 
times referred to as a surround signal), and is used 
for providing the listener with presence and sound 
expansion. The output difference signal D(n) is 
branched into four signals (S401 - S404). 

25 Among the four branched signals of the dif- 
ference signal D(n), the signal S401 is input Into 
the delay element 53-1 where it is delayed by t1. 
The delayed signal S401 is applied to the multiplier 
54-1. The multiplier 54-1 multiplies the signal S401 

30 by the coefficient g1 so as to adjust the gain. The 
resulting signal S411 is output to the adder 56-1. 
Similarly, the signal S404 is input into the delay 
element 53-5 where it is delayed by t2. and then 
input into the delay element 53-6 where it is de- 

35 layed by t1. The delayed signal S404 is applied to 
the multiplier 54-4. The multiplier 54-4 multiplies 
the delayed signal S404 by a coefficient g4 so as 
to adjust the gain. The resulting signal S414 is 
output to the adder 56-2. 

40 Herein, the delay time t1 which is common to 

the two signals (referred to as signals in a second 
system) is a delay time to delay the second-sys- 
tem signals with respect to the first-system signals 
which are processed by the FIR filters 55-1 - 55-4. 

45 That is. the second-system signals are reproduced 
with a time difference from the first-system signals 
(i.e., delayed by t1). The delay time r1 can be set 
to be, for example, about 20 msec. 

The delay time t2 is set such that, when the 

50 second-system signals S411 and S414 are repro- 
duced from the reproducing loudspeakers 57-1 and 
57-2, the reproduced signals simultaneously reach 
the third listen r 58-3 who stays at the position 
shifted to the right from the center. That is, t2 is 

55 set so as to correct the effects of the difference 
between distances from the respective reproducing 
loudspeakers 57-1 and 57-2 to the third listener 58- 

3 (the difference between the times at which the 
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signals reach the listener and the levels of the 
Signals). Preferably, the value of t2 is usually set to 
b 1 msec, or less. 

For example, a time required for the signal 
S411 reproduced from the loudspeaker 57-1 to 
reach the third listener 58-3 is represented by ti . 
and a time required for the signal S414 reproduced 
from the loudspeaker 57-2 to reach the third lis- 
tener 58-3 is represented by t2 (where ti and ta are 
assumed to be discrete times). The signal S411 
received by the third listener 58-3 is expressed as 
a1»g1 •D(n-Tl-ti), and the signal S414 is ex- 
pressed as )81 •g4»D(n-Tl-T2-t2), where a1 and ^1 
denote the attenuation of levels of reached signals 
depending on the distance. 

By setting the delay time r2 by the delay 
element 53-5 so as to satisfy the condition that t2 
= ti - 12. and setting the gain g4 of the multiplier 
54-4 so as to satisfy the condition that g4 = 
(al/)91) • g1. the third listener 58-3 can receive the 
two sounds reproduced from the loudspeakers 57-1 
and 57-2 at the equal levels. As a result, the 
presence and the expansion can be effectively 
provided for the third listener 58-3 at the position 
shifted to the right from the center. 

Alternatively, the sign of the gain g4 may be 
inverted from the sign of the gain g1, so that g4 = 
-(a1/i31) • g1. In such a case, the third listener 58- 
3 receives the difference signal D(n) from the 
speaker 57-2 in anti-phase. Thus, greater effects 
can be attained. 

Accordingly, although the third listener 58-3 
cannot feel the expansion as the result of the 
sound image control for the first-system signals 
using the FIR filters 55-1 - 55-4, the third listener 
58-3 can feel spatial expansion by reproducing the 
second-system difference signal D(n) including re- 
verberation components of the stereo signals. 

On the other hand, among the branched sig- 
nals of the difference signal D(n). the signal S403 
is input into the delay element 53-4 where it is 
delayed by r3. The delayed signal S403 is applied 
to the multiplier 54-3. The multiplier 54-3 multiplies 
the delayed signal S403 by a coefficient g3, so as 
to adjust the gain. The resulting signal S413 is 
output to the adder 56-2. Similarly, the signal S402 
is input into the delay element 53-2 where it is 
delayed by t4. and then input into the delay ele- 
ment 53-3 where it is delayed by t3. The delayed 
signal S402 is applied to the multiplier 54-2. The 
multiplier 54-2 multiplies the delayed signal S402 
by a coefficient g2, so as to adjust the gain. The 
r suiting signal S412 is output to the adder 56-1. 

Herein, th delay time t3, which is common to 
the two signals (referred to as signals in a third 
syst m), is a delay time to delay the third-system 
signals with respect to the first-system signals 
which are processed by the FIR filters 55-1 - 55-4. 



That is, the third-system signals are reproduced 
with a respective time difference from the first- 
system and second-system signals (i.e., delayed 
by t3 and t3-t1). 

5 The delay time t3 can be set to be, for exam- 

ple, about 30 msec. The delay time t4 is set such 
that, when the third-system signals S412 and S413 
are reproduced from the reproducing loudspeakers 
57-1 and 57-2, the reproduced signals simulta- 

10 neously reach the second listener 58-2 who stays 
at the position shifted to the left from the center. 
That is. t4 is set so as to correct the effects of the 
difference between distances from the respective 
reproducing loudspeakers 57-1 and 57-2 to the 

75 second listener 58-2 (the difference between times 
at which the signals reach the listener and the 
levels of the signals). Preferably, the value of t4 is 
usually set to be 1 msec, or less. 

For example, a time required for the signal 

20 S412, reproduced from the loudspeaker 57-1 to 
reach the second listener 58-2, is represented by 
ta. and a time required for the signal S413, repro- 
duced from the loudspeaker 57-2 to reach the 
second listener 58-2, is represented by t* (where. 

25 ta and t4 are assumed to be discrete times). The 
signal S412 received by the second listener 58-2 is 
expressed as a2«g2«D(n-T3-T4-t3), and the signal 
S413 is expressed as i92*g3»D(n-T3-t4). where a2 
and 02 denote the attenuation of levels of reached 

30 signals depending on the distance. 

By setting the delay time t4 by the delay 
element 53-2 so as to satisfy the condition that t3 
= t4 - ta. and setting the gain g2 of the multiplier 
54-2 so as to satisfy the condition that g2 = 

35 (02/a2) • g3. the second listener 58-2 can receive 
the two sounds reproduced from the loudspeakers 

57- 1 and 57-2 at the equal levels. As a result, the 
presence and the expansion can be effectively 
provided for the second listener 58-2 at the posi- 

40 tion shifted to the left from the center. 

Alternatively, the sign of the gain g2 may be 
inverted from the sign of the gain g3.-so that g2 = 
'(02/a2) • g3. In such a case, the second listener 

58- 2 receives the difference signal D(n) from the 
45 speaker 57-1 in anti-phase. Thus, greater effects 

can be attained. 

Accordingly, although the second listener 58-2 
cannot feel the expansion as the result of the 
sound image control for the first-system signals 

50 using the FIR filters 55-1 - 55-4. the second listener 
58-2 can feel spatial expansion by reproducing the 
third-system difference signal D(n) including rever- 
beration components of the stereo signals. 

The respective signals are added by the ad- 

55 ders 56-1 and 56-2 in the following manner, and 
reproduced from the loudspeakers 57-1 and 57-2. 
The adder 56-1 adds the output signals S501 and 
S503 from the FIR filters 55-1 and 55-3 and the 



35 



EP 0 684 751 A1 



36 



output signals S411 and S412 from the multipliers 
54-^1 and 54-2. so as to output the added signal 
S601. The added signal S601 is reproduced from 
the reproducing loudspeaker 57-1. Similarly, the 
adder 56-2 adds the output signals S502 and S504 
from the FIR filters 55-2 and 55-4, and the output 
signals S413 and S414 from the multipliers 54-3 
and 54-4. so as to output the added signal S602. 
The added signal S602 is reproduced from the 
reproducing loudspeaker 57-2. 

By adjusting the ratio of addition in the adders 
56-1 and 56-2, it is possible to determine which 
one of the listeners 58-1 - 58-3 can receive the 
sound in the best condition. For example, if the 
signals S412 and S413 are added at a larger ratio, 
the deterioration of the optimal sound for the sec- 
ond listener 58-2 can be reduced. The signals by 
which the second listener 58-2 can receive the 
sound in the best condition are the signals which 
are localized forwardly for the first and third listen- 
ers 58-1 and 58-3. Similarly, the optimal signals for 
the first listener 58-1 are the signals which are 
localized forwardly for the second and third listen- 
ers 58-2 and 58-3. and the optical signals for the 
third listener 58-3 are the signals which are lo- 
calized fonwardly for the first and second listeners 
58-1 and 58-2. 

As described above, according to this example, 
even in the case where there are three listeners, all 
of th listeners can feel expansion and presence. 
Specifically, the sound image control using the FIR 
filtering process is adopted for the listener at the 
center position, and the reproduction by delaying 
the difference signal including reverberation com- 
ponents is adopted for the listeners at the left and 
right positions, whereby offering the expansion and 
presence to all of the listeners. 

In general, the difference signals D(n) of the 
stereo audio signals include, as large components, 
reverberation sound and sounds which are not re- 
quired to be clearly localized at the center of the 
reproducing loudspeakers. By causing such dif- 
ference signals D(n) to be received in anti-phase, 
the listeners can obtain a vague expansion feeling 
without clearly localized position of the sound im- 
age and a feeling surrounded by reverberation 
sound. In general, if the listeners receive only the 
sound in anti-phase, the listeners may have a 
strange feeling due to the sound anti-phased too 
strongly. However, according to the invention, the 
r spective listeners receive normal-phased sounds 
as well as sounds in anti-phase, so that the listen- 
ers can naturally feel expansion and presenc . 

In this example, the difference signal is 
branched into four signals for th case where two 
listeners stay at off-center positions. The present 
invention is not limited to this specific case. Alter- 
natively, the difference signal may be branched 



into five or more signals for the case where two or 
more listeners stay at off-center positions. In such 
a case, the delay and multiplication processes may 
perform in the same way as those described 
5 above. 

In this example, two reproducing loudspeakers 
are used. In another case where three or more 
reproducing loudspeakers are used, a pair of loud- 
speakers may be used for a listener so as to 

70 localize the sound image at the expanded position 
from the loudspeakers, and another pair of loud- 
speakers may be used for another listener so as to 
output the difference signal of the stereo audio 
signals in anti-phase. 

15 In this example, the filter coefficients are deter- 

mined so as to localize the sound image at the 
expanded position from the reproducing loud- 
speakers with respect to the first listener. The 
present invention is not limited to such determina- 

20 tion. Alternatively, the filter coefficients may be 
determined so as to localize the sound image in 
front of or in the rear of the first listener. 

Example 5 

25 

Next, a sound field and sound image control 
apparatus, and a sound image control method ac- 
cording to a fifth example of the invention will be 
described. This example describes an apparatus 
30 which provides expansion and presence for a plu- 
rality of listeners and which can improve the clarity 
of speech when input signals include speech sig- 
nals. 

Figure 13 is a block diagram showing the 

35 sound field and sound image control apparatus 700 
in the fifth example. 

The apparatus 700 includes stereo signal input 
terminals 51-1 and 51-2, a subtracter 52, delay 
elements 53-1 - 53-6, multipliers 54-1 - 54-4, FIR 

40 filters 55-1 - 55-4. adders 56-1 and 56-2, and 
reproducing loudspeakers 57-1 and 57-2. Through 
the stereo signal input terminals 51-1. and 51-2, 
stereo signals SL(n) and SR(n) are input. The sub- 
tracter 52 calculates a difference between the ste- 

45 reo signals SL(n) and SR(n), so as to obtain a 
difference signal D(n). Each of the delay elements 
53-1 - 53-6 receives a corresponding branched 
difference signal D(n), and delays the signal by a 
predetermined time. The times delayed by the 

50 delay elements 53-1 - 53-6 are respectively pre- 
determined. The multipliers 54-1 - 54-4 perform the 
gain adjustment by multiplying the delayed dif- 
ference signals D(n) by respective predetermined 
coefficients (g1 - g4). The FIR filters 55-1 - 55-4 

55 perform the filtering process to th stereo signals 
SL(n) and SR(n) (the filter coefficients H1(n) - H4- 
(n)). The adders 56-1 and 56-2 add the outputs 
from the FIR filters 55-1 - 55-4 and the outputs 



37 



EP 0 684 751 A1 



38 



from the multipliers 54-1 - 54-4. The reproducing 
fbudspeakers 57-1 and 57-2 reproduce the output 
signals from the adders 56-1 and 56-2. 

The apparatus 700 further includes direct 
sound adders 61-1 and 61-2 for adding the stereo 
signals SL(n) and SR(n) input through the stereo 
signal input terminals 51-1 and 51-2 to the . output 
signal S601 of the adder 56-1 and the output signal 
S602 of the adder 56-2, respectively. 

As in the fourth example, a first listener 58-1 
stays at a center position in front of the two re- 
producing loudspeakers 57-1 and 57-2. A second 
listener 58-2 stays on the left side of the first 
listener 58-1. A third listener 58-3 stays on the right 
side of the first listener 58-1. 

In the apparatus 700 with the- above construc- 
tion, the output signal S601 of the adder 56-1 and 
the stereo signal SL(n) are added by the direct 
sound adder 61-1 which is connected to the output 
of the adder 56-1. and then reproduced from the 
reproducing loudspeaker 57-1. Also, the output sig- 
nal S602 of the adder 56-2 and the stereo signal 
SR(n) are added by the direct sound adder 61-2 
which is connected to the output of the adder 56-2. 
and then reproduced from the reproducing loud- 
speaker 57-2. 

The remaining operations are the same as 
those described in the fourth example shown in 
Rgure 12. 

According to the apparatus 700 of this exam- 
ple, the reproduction is performed by adding the 
direct sound to the signals S601 and S602 which 
are processed for the sound image control and the 
presence creation, whereby the clarity of speech 
can be improved while the expansion and presence 
are maintained. 

As described above, according to the sound 
field and sound image control apparatus of the 
invention, the reproduction with expansion for the 
listener positioned at the center is provided by 
localizing the sound image at a position other than 
the positions of the reproducing loudspeakers, and 
the reproduction with expansion for the listeners at 
positions shifted from the center is provided by 
outputting difference signals of the stereo audio 
signals. Therefore, the listener's positions are not 
limited in the center of the sound field and sound 
image control apparatus, and the audio reproduc- 
tion with expansion can be performed in a wide 
service area. 

Various other modifications will be apparent to 
and can be readily made by those skilled in the art 
without departing from the scop and spirit of this 
invention. Accordingly, it is not intended that the 
scope of the claims append d hereto be limited to 
th description as set forth herein, but rather that 
th claims be broadly construed. 



Claims 

1. An apparatus which calculates filter coefficients 
for controlling sound field and sound image, 

5 based on a plurality of first impulse response 

signals and a pair of second impulse response 
signals, the plurality of first impulse response 
signals indicating impulse responses from 
loudspeakers reproducing audio signals to both 

10 ears of a listener, the pair of second impulse 

response signals indicating impulse responses 
from a reference loudspeaker at a position at 
which a sound image is localized to both ears 
of the listener, the apparatus comprising: 

75 feature extracting means for receiving the 

pair of second impulse response signals, for 
extracting parameters representing features of 
the pair of second impulse response signals, 
and for outputting parameter signals; 

20 signal adjusting means for adjusting at 

least one of the plurality of first impulse re- 
sponse signals based on the parameter sig- 
nals, and for outputting a pair of third impulse 
response signals having the same features as 

25 the extracted features; and 

coefficient calculation means for calculat- 
ing the filter coefficients for controlling the 
sound field and sound image, based on the 
plurality of first impulse response signals and 

30 the pair of third impulse response signals ap- 

plied from the signal adjusting means. 

2. An apparatus according to claim 1 , wherein the 
coefficient calculation means sets the filter co- 

35 efficients so that the pair of third impulse re- 

sponse signals are substantially equal to a pair 
of fourth impulse response signals,, the pair of 
fourth impulse response signals indicating a 
pair of impulse responses at both ears of the 

40 listener when impulse signals are reproduced 

from the reproducing loudspeakers. 

3- An apparatus according to claim 2, further 
comprising: 

45 response characteristic calculation means 

for calculating a pair of impulse responses at 
both ears of the listener when the impulse 
signals are reproduced from the reproducing 
loudspeakers, based on the first impulse re- 

50 sponse signals and the filter coefficients, and 

for outputting the pair of fourth impulse re- 
sponse signals; 

comparison means for comparing the pair 
of fourth impulse r spons signals with the pair 

55 of third impulse response signals, and for out- 

putting a correlation signal; and 

control means for outputting a control sig- 
nal which controls the coefficient calculation 



EP 0 684 751 A1 



40 



means, based on the correlation signal. 
* wherein, in accordance with the control 
signal, the coefficient calculation means selec- 
tiv ly performs one of two operations, in one 
operation signals indicative of the calculated 
filter coefficients are output, and in th other 
operation the filter coefficients are again cal- 
culated using signals which are obtained by 
delaying the pair of third impulse response 
signals by a predetermined time. 

4. An apparatus according to claim 1 , wherein the 
feature extracting means comprises: 

level ratio detection means for receiving 
the pair of second impulse response signals, 
for detecting a level ratio a of the pair of 
second impulse response signals, and for out- 
putting a level ratio detection signal; and 

time difference detection means for receiv- 
ing the pair of second impulse response sig- 
nals, for detecting a time difference dt of the 
pair of second impulse response signals, and 
for outputting a time difference detection sig- 
nal. 

5. An apparatus according to claim 4, wherein the 
signal adjusting means comprises: 

selecting means for selecting a pair of first 
impulse response signals from among the plu- 
rality of first impulse response signals; 

time difference adjusting means for receiv- 
ing the selected pair of first impulse response 
signals and the time difference detection sig- 
nal, for adjusting the selected pair of first im- 
pulse response signals so that a relative time 
difference of the pair of first impulse response 
signals is equal to the time difference dt based 
on the time difference detection signal, and for 
outputting a pair of adjusted impulse response 
signals; and 

level ratio adjusting means for receiving 
the pair of adjusted impulse response signals 
and the level ratio detection signal, for adjust- 
ing a gain of the pair of the adjusted impulse 
response signals so that the level ratio of the 
adjusted impulse response signals in the pair 
is equal to the level ratio '^[^Ka'^[ based on 
the level ratio detection signal, and for output- 
ting the pair of gain-adjusted signals as the 
pair of third impulse response signals. 

6. An apparatus according to claim 4. wherein the 
signal adjusting means comprises: 

selecting means for selecting one first im- 
pulse response signal from among the plurality 
of first impulse response signals; 

time difference adjusting means for receiv- 
ing the selected first impulse response signal 



and the time difference detection signal, for 
delaying the selected first impulse response 
signal by the time difference dt based on the 
time difference detection signal, and for out- 

5 putting a delayed impulse response signal; and 

level ratio adjusting means for receiving 
the delayed impulse response signal and the 
level ratio detection signal, for adjusting a gain 
of the delayed impulse response signal by 

70 multiplication of the delayed impulse response 

signal by the level ratio ''['^Ka^[ based on 
the level ratio detection signal, and for output- 
ting an adjusted impulse response signal, and 
wherein the pair of third impulse response 

75 signals are constituted of the selected first 

impulse response signal and the adjusted im- 
pulse response signal. 

7- An apparatus according to claim 1 . wherein the 
20 feature extracting means is transfer character- 

istic detection means for receiving the pair of 
second impulse response signals, for detecting 
transfer characteristics of the pair of second 
impulse response signals, for calculating a 
25 transfer characteristic ratio, and for outputting a 

characteristic ratio signal. 

8, An apparatus according to claim 7. wherein the 
signal adjusting means comprises: 

30 selecting means for selecting one first im- 

pulse response signal from among the plurality 
of first impulse response signals; and 

transfer characteristic adjusting means for 
receiving the selected first impulse response 

35 signal and the characteristic ratio signal, for 

adjusting a transfer characteristic of the se- 
lected first impulse response signal based on 
the characteristic ratio, and for outputting an 
adjusted impulse response signal, and 

40 wherein the pair of third impulse response 

signals are constituted of the selected first 
impulse response signal and the adjusted im- 
pulse response signal. 

45 9. An apparatus according to claim 8, wherein 

the transfer characteristic detection means 
comprises: first transform means for transform- 
ing the received pair of second impulse re- 
sponse signals into a pair of first characteristic 

50 signals represented in frequency domain; and 

first calculation means for calculating a transfer 
characteristic ratio of the pair of second im- 
pulse response signals based on the first char- 
acteristic signals, and 

55 the transfer characteristic adjusting means 

comprises: second transform means for trans- 
forming the selected first impulse response 
signal into a second characteristic signal repre- 



41 



EP 0 684 751 A1 



42 



sented in frequency domain; second calcula- 
tion nneans for nnultiplying the second char- 
acteristic signal by the transfer characteristic 
ratio indicated by the characteristic ratio signal; 
and inverse transform means for transforming 
the multiplied signal into a signal represented 
in time domain. 

10- An apparatus according to claim 9. wherein the 
first and second transform means are Fourier 
transform means, and the inverse transform 
means is inverse Fourier transform means. 

11. A sound field/sound image control apparatus 
which performs a sound field control and a 
sound image localization by processing stereo- 
phonic signals including a plurality of channel 
signals, the apparatus comprising: 

input means for inputting the plurality of 
channel signals; 

first signal processing means for receiving 
the plurality of channel signals, for performing 
a filtering process after dividing each of the 
channel signals into a plurality of branched 
signals, and for outputting a plurality of first 
processed signals; 

subtracting means for receiving at least 
two of the plurality of channel signals, for pro- 
ducing a difference signal by subtracting one 
of the two channel signals from the other chan- 
nel signal, and for outputting the difference 
signal; 

at least one pair of second signal process- 
ing means, each for receiving the difference 
signal, for delaying the difference signal by a 
predetermined time, for adjusting the level to a 
predetermined level, and for outputting a pair 
of second processed signals; 

at least one pair of adding means for re- 
ceiving the first processed signals and at least 
a pair of the second processed signals, for 
adding the first and the second processed 
signals at a predetermined ratio, and for out- 
putting at least a pair of added signals; and 

at least one pair of reproducing means, 
each for receiving a corresponding one of the 
added signals, and for reproducing the cor- 
responding signal at a predetermined position, 

wherein the sound image is localized by 
reproducing the first processed signals, and 
the sound field is reproduced with presence by 
reproducing the second processed signals. 

12. An apparatus according to claim 11, wherein 
the pair of the second signal proc ssing 
means comprises: 

first delay means for delaying both of the 
received pair of difference signals by a pre- 



determined time with respect to the first pro- 
cessed signals; 

second delay means for delaying one of 
the pair of difference signals by a predeter- 
5 niined time with respect to the other difference 

signal; and 

multiplying means for multiplying the pair 
of difference signals by respective predeter- 
mined coefficients. 

10 

13. An apparatus according to claim 12. wherein 
the predetermined coefficients, which are mul- 
tiplied to the pair of difference signals, have 
reversed signs from each other, whereby one 

75 of the pair of difference signals is an anti- 

phase signal of the other difference signal. 

14. An apparatus according to claim 12, wherein 
the predetermined delay time used in the sec- 

20 ond delay means is set based on a reach time 

difference between a pair of signals which 
reach a listener from at least the pair of re- 
producing means, whereby the listener simul- 
taneously receives the signals from at least the 

25 pair of reproducing means. 

15. An apparatus according to claim 11. further 
comprising second adding means for receiving 
the pair of added signals and the two channel 

30 signals, for adding one of the pair of added 

signals to one of the two channel signals, and 
for adding the other added signals to the other 
channel signals. 

35 16. A method for calculating filter coefficients for 
controlling sound field and sound image, 
based on a plurality of first impulse response 
signals and a pair of second impulse response 
signals, the plurality of first impulse response 

40 signals indicating impulse responses from 

loudspeakers reproducing audio signals to both 
ears of a listener, the pair of second impulse 
response signals indicating impulse responses 
from a reference loudspeaker at a position at 

45 which a sound image is localized to both ears 
of the listener, the method comprising the 
steps of: 

(a) extracting features of the pair of second 
impulse response signals, and producing a 

50 parameter signals representing the features; 

(b) adjusting at least one of the plurality of 
first impulse response signals based on the 
parameter signals, and producing a pair of 
third impulse response signals having the 

55 same features as the extracted features; 

and 

(c) calculating the filter coefficients for con- 
trolling the sound field and sound image. 
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. based on the plurality of first impulse re- 
4 sponse signals and the produced pair of 
third impulse response signals. 

17. A method according to claim 16, wherein in 
the step (c). the filter coefficients are set so 
that the pair of third impulse response signals 
are substantially equal to a pair of fourth im- 
pulse response signals, the pair of fourth im- 
pulse response signals indicating a pair of im- 
pulse responses at both ears of the listener 
when impulse signals are reproduced from the 
reproducing loudspeakers. 

18- A method according to claim 17, further com- 
prising the steps of: 

(d) calculating a pair of impulse responses 
at both ears of the listener when the im- 
pulse signals are reproduced from the re- 
producing loudspeakers, based on the first 
Impulse response signals and the filter co- 
efficients, and producing the pair of fourth 
impulse response signals; 

(e) comparing the pair of fourth impulse 
response signals with the pair of third im- 
pulse response signals, and producing a 
correlation signal; and 

(f) producing a control signal which controls 
the coefficient calculation, based on the cor- 
relation signal, 

wherein, in the step (c), in accordance 
with the control signal, one of step (c1) of 
producing signals indicative of outputting 
the calculated filter coefficients and step 
(c2) of calculating again the filter coeffi- 
cients using signals which are obtained by 
delaying the pair of third impulse response 
signals by a predetermined time. 

19. A method according to claim 16, wherein step 

(a) comprises the steps of: 

(a1) detecting a level ratio a of the pair of 
second impulse response signals, and pro- 
ducing a level ratio detection signal; and 
(a2) detecting a time difference dt of the 
pair of second impulse response signals, 
and producing a time difference detection 
signal. 

20. A method according to claim 19, wherein step 

(b) comprises the steps of: 

(b1) selecting one pair of first impulse re- 
sponse signals from among the plurality of 
first impulse r sponse signals; 
(b2) adjusting the pair of first impuls re- 
sponse signals so that a relative time dif- 
ference of the pair of first impulse r sponse 
signals is equal to the time difference dt 
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based on the time difference detection sig- 
nal, and producing a pair of adjusted im- 
pulse response signals; and 
(b3) adjusting a gain of the pair of the 
adjusted impulse signals so that the level 
ratio of the adjusted impulse response sig- 
nals in the pair is equal to the level ratio a 
based on the level ratio detection signal, 
and producing the pair of gain-adjusted sig- 
nals as the pair of third impulse response 
signals. 



21. A method according to claim 19. wherein step 
(b) comprises the steps of: 

75 (b4) selecting one first impulse response 

signal from among the plurality of first im- 
pulse response signals; 
(b5) delaying the selected first impulse re- 
sponse signal by the time difference dt 

20 based on the time difference detection sig- 

nal, and producing a delayed impulse re- 
sponse signal; and 

(b6) adjusting a gain of the delayed impulse 
response signal by multiplying the delayed 

25 impulse response signal by the level ratio a 

based on the level ratio detection signal, 
and producing an adjusted impulse re- 
sponse signal, and 

wherein the pair of third impulse re- 

30 sponse signals are constituted of the se- 

lected first impulse response signal and the 
adjusted impulse response signal. 

22. A method according to claim 16. wherein step 
35 (a) comprises the steps of (a3) detecting trans- 
fer characteristics of the pair of second im- 
pulse response signals, and (a4) calculating a 
transfer characteristic ratio, and producing a 
characteristic ratio signal. 

40 

23. A method according to claim 22. wherein step 
(b) comprises the steps of: 

(b7) selecting one first impulse response 
signal from among the plurality of first im- 

45 pulse response signals; and 

(b8) adjusting a transfer characteristic of the 
selected first impulse response signal 
based on the characteristic ratio, and pro- 
ducing an adjusted impulse response sig- 

50 nal, and 

wherein the pair of third impulse re- 
sponse signals are constituted of the se- 
lected first impulse response signal and the 
adjusted impuls response signal. 

55 

24. A method according to claim 23, wher in 

step (a3) comprises: a first transform step 
of transforming the received pair of second 
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* impulse response signals into a pair of first 
*^ characteristic signals represented in frequency 
domeiin; and a first calculation step of calculat- 
ing a transfer characteristic ratio of the pair of 
second impulse response signals based on the 
first characteristic signals, and 

step (b8) comprises: a second transform 
step of transforming the selected first impulse 
response signal into a second characteristic 
signal represented in frequency domain; a sec- 
ond calculation step of multiplying the second 
characteristic signal by the transfer character- 
istic ratio indicated by the characteristic ratio 
signal; and an inverse transform step of trans- 
forming the multiplied signal into a signal re- 
presented in time domain. 

25. A method according to claim 24. wherein in 
the first and second transform steps, Fourier 
transforms are performed, and in the inverse 
transform step, an Inverse Fourier transform is 
performed. 

26. A sound field/sound image control method for 
perfomning a sound field control and a sound 
image localization by processing stereophonic 
signals including a plurality of channel signals, 
the method comprising: 

an input step of inputting the plurality of 
channel signals; 

a first signal processing step of performing 
a filtering process after dividing each of the 
channel signals into a plurality of branched 
signals, and producing a plurality of first pro- 
cessed signals; 

a subtracting step of subtracting one of at 
least two of the plurality of channel signals 
from the other channel signal, and producing a 
difference signal; 

a second signal processing step of delay- 
ing the difference signal by a predetermined 
time, adjusting the level to a predetermined 
level, and producing a pair of second pro- 
cessed signals; 

an adding step of adding the first pro- 
cessed signals and at least a pair of the sec- 
ond processed signals at a predetermined ra- 
tio, and producing at least a pair of added 
signals; and 

a reproducing step of reproducing the pair 
of added signals at predetermined positions, 

wherein the sound image is localized by 
reproducing the first processed signals, and 
the sound field is reproduced with presence by 
reproducing the second processed signals. 

27. A method according to claim 26. wherein the 
second signal processing step includes: 



a first delay step of delaying both of the 
received pair of difference signals by a pre- 
determined time with respect to the first pro- 
cessed signals; 
5 a second delay step of delaying one of the 

pair of difference signals by a predetermined 
time with respect to the other difference signal; 
and 

a multiplying step of multiplying the pair of 
70 difference signals by respective predetermined 

coefficients. 

28. A method according to claim 27. wherein the 
predetermined coefficients which are multiplied 
75 to the pair of difference signals have reversed 

signs from each other, whereby one of the pair 
of difference signals is an anti-phase signal of' 
the other difference signal. 

20 29. A method according to claim 27. wherein the 
predetermined delay time used in the second 
delay step is set based on a reach time dif- 
ference between the pair of added signals re- 
produced in the reproducing step which reach 

25 a listener, whereby the listener simultaneously 

receives the reproduced pair of added signals. 

30. A method according to claim 26. further com- 
prising a second adding step of adding one of 
30 the pair of added signals to one of the two 
channel signals, and for adding the other ad- 
ded signals to the other channel signals. 
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